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1.  Introduction 


Accurate  matching  and  measurement  of  breast  lesions  identified  on  multiple  temporal  mammographic 
views  of  breast  is  vital  in  detecting  and  treating  breast  cancer.  However,  lack  of  3D  structural  knowledge 
and  large  compression  of  breast  during  X-ray  imaging  often  cause  mismatch  among  temporal 
mammograms,  which  eventually  leads  to  incorrect  diagnosis  or  localization.  A  3D  model  is  strongly 
desired  that  can  provide  accurate  information  about  breast’s  3D  geometry  as  well  as  its  deformation 
during  mammographic  imaging.  Development  of  a  3D  finite  element  model  is  proposed  to  simulate  and 
analyze  breast  deformation  that  can  significantly  improve  the  accuracy  of  matching  in  temporal 
mammograms  and  thus  the  performance  of  diagnosis  and  treatment. 

In  this  research  project,  we  have  developed  mathematical  model  of  breast  deformation  to  simulate  breast 
compression  during  mammographic  imaging.  We  have  developed  two  types  of  mammogram  registration 
methods:  magnetic  resonance  imaging  (MRI)  guided  registration  and  generic  registration.  Datasets  have 
been  constructed  to  validate  the  developed  registrations  methods.  Mammogram  registrations  have  been 
performed  on  concurrent  mammograms  using  MRI  guided  and  generic  registration  method,  respectively. 
Also  both  registration  methods  have  been  applied  onto  temporal  mammograms.  Promising  evaluation 
results  demonstrate  feasibility  of  the  developed  mammogram  registration  methods.  The  results  also 
demonstrate  that  the  developed  registration  methods  would  be  very  helpful  in  mammogram  interpretation 
for  improving  accuracy  of  detection  and  diagnosis  of  breast  cancer. 


2.  Body 

Breast  deformation  during  mammography  has  been  modeled  using  displacement  differential  equation 
together  with  boundary  conditions,  and  breast  compression  procedure  has  been  simulated  using  rigorous 
finite  element  analysis  (FEA)  algorithm.  Based  on  the  developed  mathematical  breast  deformation  model 
and  simulations,  two  types  of  breast  deformation  model  have  been  developed:  magnetic  resonance  image 
(MRI)  guided  deformation  model  and  generic  deformation  model. 


2.1  Mathematical  model  and  simulations  of  breast  deformation  in  mammography 

The  female  breast  is  essentially  composed  of  four  structures:  lobules  or  glands,  milk  ducts,  fat,  and 
connective  tissue.  Most  biologic  tissues  have  both  a  viscous  and  an  elastic  response  to  external 
deformations.  Since  we  are  interested  only  in  slow  deformations,  the  response  of  the  tissue  can  be 
considered  entirely  due  to  elastic  forces. 


Under  assumptions  that  breast  is  uncompressible  with  isotropic  property,  breast  deformation  can  be 
described  by  the  following  differential  equations  named  Navier  equations: 
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The  above  displacement  differential  equations  have  been  solved  using  numerical  method,  finite  element 
analysis  (FEA)  algorithm.  (FEA)  algorithm  is  the  most  accurate  numerical  technique  in  modeling 
deformable  objects.  Its  physical  soundness  and  mathematical  rigor  ensures  the  accuracy  of  breast 
deformation  modeling  that  is  essential  for  multiple  mammograms  interpretation. 
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The  basic  steps  of  the  FEM  approach  involving  object  deformations  are  the  following:  (1)  Derive  an 
equilibrium  equation  for  the  continuum  with  given  material  properties;  (2)  Select  the  appropriate  finite 
elements  and  corresponding  interpolation  functions  (also  called  shape  functions )  for  the  problem;  (3) 
Subdivide  the  object  into  the  elements  (meshing);  (4)  Obtain  the  stiffness  matrices  for  each  element;  (5) 
Assemble  the  global  stiffness  matrix  using  the  element  stiffness  matrices;  (6)  Impose  the  given  boundary 
conditions;  (7)  Solve  the  system  of  equations  for  the  vector  of  unknown  variables.  We  used  the 
commercial  software  AN SYS,  for  numerically  solving  the  partial  differential  equations.  The  resulting 
meshed  volume  and  simulations  of  breast  deformation  are  presented  in  Figure  1. 


a 


b 


c 


Fig.  1 .  Mathematical  modeling  and  simulations  of  breast  deformation.  (a)Meshing  on  breast  volume  using  FEA 
algorithm;  (b)  Simulation  of  CC  view  compression  (white  color:  natural  breast  shape  before  compression,  blue 
color:  breast  shape  after  compression)  (c)  Simulation  of  ML  view  compression  (white  color:  natural  breast  shape 
before  compression;  blue  color:  breast  shape  after  compression) 

2.2  Development  of  registration  model  for  mammogram  registration 

Two  types  of  mammogram  registration  model  have  been  developed:  magnetic  resonance  image  (MRI) 
guided  registration  model  and  generic  registration  model.  MRI  guided  registration  model  is  developed  for 
cases  that  concurrent  MR  images  are  available  with  mammograms.  For  most  cases  that  only 
mammograms  are  available,  generic  registration  model  has  been  developed. 

2.2.1  MRI  guided  registration  model 

MR  images  are  applied  to  construct  breast  volume.  In  each  MRI  slice,  breast  is  segmented  and  the  2D 
breast  contour  is  extracted  using  standard  morphological  operators.  B -spline  smoothing  is  implemented  to 
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remove  small  sharp  edges  that  might  have  been  generated  during  segmentation.  Then,  the  3D  breast  shape 
is  constructed  by  combining  all  the  2D  breast  contours  as  shown  in  Figure  2. 


(c)  (d) 

Figure  2,  Natural  breast  volume  construction  using  MR  images,  (a)  Patient  MRI  slices  for  building  the  3D  model,  (b)  Wire 
Frame  of  the  Breast,  (c)  Non  planar  area  fitted  on  spline  curves  (Coons  patch),  (d)  Constmcted  breast  volume. 


2.2.2  Generic  registration  model 

The  proposed  generic  registration  model  adjusts  a  generic  3D  FE  breast  model  based  on  patient 
mammograms  and  the  known  compression  magnitude  and  direction  recorded  in  mammograms.  The  initial 
generic  model  was  selected  from  the  library  of  existing  patient  MR  volumes  based  on  the  criterion  of 
similar  proportion.  The  mammograms  were  compared  to  the  generic  model  projections  to  obtain  several 
scaling  factors.  The  three  dimensional  scaling  factors  were  computed  using  the  ratio  of  patient  breast  and 
projection  in  x,  y,  and  z  directions.  The  following  equations  were  used  to  calculate  the  scaling  factors. 
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A  new  model  was  constructed  using  the  adjusted  contour  and  the  projection  error  based  on  overlapping 
area  was  calculated.  The  above  process  is  performed  using  several  initial  generic  models  and  the  one  with 
the  minimum  error  rate  is  chosen  as  generic  model  for  patient. 
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Complete  scaling  algorithm  is  as  follows:  First  we  perform  initial  scaling  of  model  using  Sx,  Sy,  Sz. 
Then  we  compress  the  model  using  compression  distance  recorded  in  patient  mammogram.  After 
compression  the  model  is  projected  onto  CC  view  and  ML  view.  The  projection  area  is  compared  with  the 
area  of  breast  region  in  the  mammogram  and  error  is  calculated  and  algorithm  automatically  selects  from 
the  set  of  models  -  the  one  which  is  best  based  on  fit  of  initial  scaling.  For  the  model  chosen,  curvature 
comparison  is  performed.  Curvature  of  the  model  is  adjusted  using  parameters  S4  and  St:  S4  for  CC  view 
and  S5  for  ML  view.  After  curvature  adjustment,  model  is  recompressed  and  projection  error  is 
calculated.  Model  scaling  with  less  projection  error  is  chosen  and  different  measures  are  calculated  on 
scaled  model.  A  typical  reconstructed  breast  volume  from  mammogram  using  generic  registration  models 
is  illustrated  in  Figure  3. 


Figure  3.  Generic  Model  construction.  (Left:)  Generic  model  before  using  scaling  factors.  (Right:)  Generic  model  after 
comparison  of  projection  and  target  mammograms. 


2.3  Registrations  among  concurrent  mammograms 

We  developed  a  non-rigid  registration  approach  with  the  following  improvements:  (1)  a  finite  element 
model  is  superior  to  a  generic  geometrical  model  in  handling  complex  shapes  and  nonrigid  deformation. 
As  a  result,  many  restrictions  associated  with  a  geometrical  model  can  be  relieved.  For  example,  the 
assumption  that  a  breast  only  deforms  within  the  cross-section  of  CC  view  is  not  needed.  The  assumption 
that  a  corresponding  curve  can  only  deform  uniformly  is  also  waived;  (2)  compres-sion  parameters  are 
corrected  by  a  global  shape  calibration  to  ensure  the  quality  of  feature  registration.  Detail  registration 
procedure  is  described  in  Appendix  2.  Registration  of  concurrent  mammograms  is  described  in  Figure  4. 

2.3  Registrations  among  temporal  mammograms 
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Temporal  Registration  Method  analyzes  mammograms  by  examining  temporal  sequences  of  images.  Such 
temporal  comparisons  have  value  because,  to  a  first  approximation,  normal  breasts  do  not  change 
significantly  over  time,  except  for  minor  variations  associated  with  the  menstrual  cycle  or  significant 
changes  in  body  weight.  Some  pathological  changes  in  the  breast  are  sufficiently  subtle  that  they  may 
pass  unnoticed  for  many  years;  thus,  radiologists  compare  images  from  a  number  of  previous  years.  Such 
changes  can  be  further  obfuscated  by  different  choices  of  X-ray  technique,  and  variation  in  breast 
positioning  or  compression.  Schematic  of  Registration  on  temporal  mammograms  is  described  in  Figures 
5. 
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Fig.  4.  Schematic  of  registration  Figure  5,  Schematic  of  registration  on  temporal  mammograms 

of  concurrent  mammograms 

2.4  Database 

A  dataset  containing  47  mammograms  of  32  patients  (15  patients  had  2  sets  of  mammograms  scanned  at 
different  times)  was  provided  by  the  Lifetime  Screening  Center,  H.  Lee  Moffitt  Cancer  Center  and 
Research  Institute.  Mammograms  were  scanned  at  a  resolution  of  75  micron  and  12  bits  per  pixel, 
containing  mass  or  calcification.  Patient  age  group  ranged  from  30  to  70  years  with  a  median  of  45  years. 
The  images  were  collected  over  a  period  of  time  from  daily  clinical  investigations.  The  center  of  a  lesion 
is  determined  as  the  center  of  a  circle  (or  an  ellipse)  manually  fitted  to  the  visible  parts  of  the  lesion  in 
patient  datasets. 

Table  1:  Database 


Patient 

Number 

Case 

Number 

Lesion  Type 

Age 

Compression 
Rate  (mm) 

With  MRI 

Lesion  visibility 

Lesion 

Number 

1 

1 

Mass 

35-50 

43 

Y 

Visible  in  both  views 

1 

2 

2 

Mass 

35-50 

46 

N 

Visible  in  both  views 

1 

3 

3 

Mass 

35-50 

49 

Y 

Visible  in  both  views 

1 

4 

4 

Mass 

50-70 

52 

N 

Visible  in  both  views 

1 

5 

5 

Mass 

50-70 

54 

N 

Visible  in  both  views 

1 

6 

6 

Mass 

35-50 

56 

N 

Visible  in  both  views 

2 

7 

7 

Mass 

35-50 

44 

N 

Visible  in  both  views 

1 

8 


8 

8 

Calcification 

50-70 

58 

Y 

Visible  in  both  views 

1 

9 

9 

Mass 

35-50 

54 

N 

Visible  in  both  views 

1 

10 

10 

Calcification 

50-70 

63 

Y 

Visible  in  both  views 

2 

11 

11 

Mass 

35-50 

45 

N 

Visible  in  both  views 

1 

12 

12 

Calcification 

50-70 

48 

N 

Visible  in  both  views 

1 

13 

13 

Calcification 

50-70 

60 

Y 

Visible  in  both  views 

2 

14 

14 

Calcification 

35-50 

52 

Y 

Visible  in  both  views 

1 

15 

15,16 

Calcification 

50-70 

47,44 

Y 

Visible  in  both  views 

1 

16 

17 

Calcification 

35-50 

61 

N 

Visible  in  both  views 

1 

17 

18 

Calcification 

50-70 

47 

N 

Visible  in  both  views 

1 

18 

19 

Calcification 

35-50 

48 

N 

Visible  in  both  views 

1 

19 

20,21 

Calcification 

50-70 

43,47 

Y 

Visible  in  both  views 

1 

20 

22,23 

Calcification 

35-50 

47,46 

Y 

Visible  in  both  views 

1 

21 

24,25 

Calcification 

50-70 

46,44 

Y 

Visible  in  both  views 

1 

22 

26,27 

Calcification 

35-50 

45,43 

Y 

Visible  in  both  views 

1 

23 

28.29 

Calcification 

50-70 

47,48 

Y 

Visible  in  both  views 

1 

24 

30,31 

Calcification 

35-50 

45,43 

Y 

Visible  in  both  views 

1 

25 

32,33 

Calcification 

50-70 

52,50 

Y 

Visible  in  both  views 

1 

26 

34,35 

Mass 

35-50 

44,43 

N 

Visible  in  both  views 

1 

27 

36,37 

Calcification 

50-70 

49,50 

N 

Visible  in  both  views 

1 

28 

38,39 

Calcification 

50-70 

45,46 

N 

Visible  in  both  views 

1 

29 

40.41 

Calcification 

35-50 

43,47 

N 

Visible  in  both  views 

1 

30 

42,43 

Mass 

50-70 

46,43 

N 

Visible  in  both  views 

1 

31 

44,45 

Calcification 

35-50 

46,49 

N 

Visible  in  both  views 

1 

32 

46,47 

Mass 

35-50 

51,50 

N 

Visible  in  both  views 

1 

2.5  Evaluation 

Several  evaluation  methods  have  been  developed  and  performed  to  test  accuracy  of  the  developed 

registration  methods. 

1.  3D  curve  distance:  we  used  the  minimum  distance  between  the  2  curves  to  measure  the  error,  the 
element  size  could  not  be  infinitely  small.  The  error  could  be  minimized  when  very  fine  elements 
were  chosen. 

2.  Predicted  3D  position:  after  the  position  of  the  matching  element  was  found  by  combining 
information  from  cc  view  and  ml  view,  we  calculated  Euclidean  distance  between  the  actual  feature 
position  in  MRI  volume  and  its  prediction. 

3.  Predicted  2D  lesion  position:  we  projected  the  element  set  which  corresponds  to  the  projection  of  the 
feature  point  in  one  view  to  the  other  view,  which  formed  a  curve,  and  then  we  calculated  the 
minimum  distance  between  the  actual  feature  point  and  the  projected  curve.  This  measure  could  also 
be  used  for  the  case  when  suspicious  area  is  visible  only  in  one  view. 

2.6  Registration  results 


Table  2.  Algorithm  performance  on  phantom  data 
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Simulated 
Mass  and 
Calcification 

Average 

Error 

(mm) 

Average 

Feature 

Diameter 

(mm) 

Average 

Feature 

Distance 

(mm) 

3D  curve 
distance 

1.6±0.4 

10- 

20 

25.9- 

80.0 

Predicted 

3D  lesion 
position 

2.6±0.8 

10- 

20 

25.9- 

80.0 

Predicted 

2D  lesion 
position 

2.1±0.4 

10- 

20 

25.9- 

80.0 

Table  3:  Registration  results  on  temporal  mammograms 
using  specific  registration  model 


Patient 

Number 

Euclidean 

distance 

(mm) 

Predicted  2D 
lesion 

position(mm) 

15 

3.2 

4.3 

19 

3.4 

4.1 

20 

3.3 

4.4 

21 

2.5 

3.7 

22 

3.4 

4.2 

23 

3.5 

4.9 

24 

4.2 

4.2 

25 

3.7 

4.5 

Average 

3.4  ±0.86 

4.3±  0.52 

Table  4:  Experiment  results:  Patient  data  using  patient  specific  models  (One  Time) 


3D  curve 
distance 
(mm) 

Predicted  2D 
lesion 

position(mm) 

Feature 

Diameter 

(mm) 

Case  1 

1.6 

3.1 

7.2 

Case  3 

1.2 

2.1 

8.1 

Case  8 

1.5 

2.6 

3.5 

Case  10 

0.4 

1.6 

5.4 

Case  13 

1.5 

2.4 

3.8 

Case  14 

1.7 

2 

5.2 

Case  15 

1.3 

2.5 

7.3 

Case  20 

2.1 

2.7 

7.1 

Case  22 

2.3 

3 

5.3 

10 


Case  24 

1.9 

2.9 

5.6 

Case  26 

2.2 

3.1 

4.8 

Case  28 

2.3 

2.9 

7.6 

Case  30 

1.9 

3 

oo 

oo 

Case  32 

2.3 

3.2 

7.5 

Average 

1.73  ±0.32 

2.65  ±  0.41 

Generic  model  algorithm  was  tested  when  we  validate  algorithm  accuracy  on  temporal  pairs.  Combined 
with  all  the  available  data  in  our  database,  results  are  listed  as  follows: 

Table  5:  Registration  results  on  temporal  mammograms 
using  generic  registration  model 


Patient 

Number 

Euclidean 

distance 

(mm) 

Predicted  2D 
lesion 

position(mm) 

15 

4.9 

5.7 

19 

5.2 

5.3 

20 

4.8 

6.9 

21 

3.9 

5.1 

22 

5.1 

5.3 

23 

4.7 

6.3 

24 

5.9 

5.1 

25 

5.8 

6.1 

Average 

5.0±  0.74 

5.7±  0.83 

Table  6:  Experiment  Results:  Patient  Data  Using  Generic  Model  (One  Time) 


Case 

3D  curve 
distance 
(mm) 

Predicted  2D 
lesion 
position 
(mm) 

Feature 

Diameter 

(mm) 

Case 

3D  curve 
distance 
(mm) 

Predicted 
2D  lesion 
position 
(mm) 

Feature 

Diameter 

(mm) 

Case  1 

3 

4.7 

7.2 

Case  25 

4.1 

4.5 

5.9 

Case  2 

5.3 

3 

10.1 

Case  26 

4.2 

5.2 

4.8 

Case  3 

4.1 

4.3 

8.1 

Case  27 

3.5 

4.7 

4.7 

Case  4 

3.9 

4.9 

7.2 

Case  28 

4.7 

3.8 

7.6 

Case  5 

3.5 

6.2 

5.2 

Case  29 

5.1 

6.2 

7.9 

Case  6 

4.1 

4.7 

2.6 

Case  30 

4.4 

3.4 

8.8 

Case  7 

4.3 

3.6 

5.1 

Case  31 

4.2 

4.4 

8.5 

Case  8 

4.6 

5.9 

3.5 

Case  32 

4.2 

4.8 

7.5 

Case  9 

4 

3.2 

2.9 

Case  33 

5.2 

5.6 

7.8 

Case  10 

3.8 

4.2 

5.4 

Case  34 

4.1 

4.8 

6.7 

Case  11 

3.8 

4.6 

3.9 

Case  35 

4.6 

5 

6.2 

11 


Case  12 

4.7 

5.3 

4.2 

Case  36 

5 

5.1 

8.2 

Case  13 

3.7 

4.6 

3.8 

Case  37 

5.1 

4.9 

8.1 

Case  14 

4.2 

4.8 

5.2 

Case  38 

5 

5.6 

7.6 

Case  15 

4.5 

4.9 

7.3 

Case  39 

4.6 

3.9 

7.4 

Case  16 

4.6 

4.7 

5.4 

Case  40 

4.9 

6.4 

8.9 

Case  17 

4.5 

5.3 

5.3 

Case  41 

4.3 

3.5 

8.5 

Case  18 

3.8 

2.4 

7.2 

Case  42 

4.1 

4.6 

7.9 

Case  19 

3.6 

2.8 

3.9 

Case  43 

4.1 

5 

8.2 

Case  20 

3.2 

4.2 

7.1 

Case  44 

5 

5.8 

8.1 

Case  21 

3.9 

4.4 

6.9 

Case  45 

4 

5 

8.3 

Case  22 

4 

5.3 

5.3 

Case  46 

4.5 

5.2 

7.1 

Case  23 

3.6 

4.6 

5.6 

Case  47 

4.7 

5.3 

6.9 

Case  24 

3.4 

4.3 

5.6 

Case  25 

4.1 

4.5 

5.9 

Average 

4.0  ±0.39 

4.5  ±0.63 

3.  Key  Research  Accomplishments 

1 .  Breast  deformation  during  mammogram  imaging  has  been  modeled  using  displacement  differential 
equation 

2.  Breast  compression  during  mammogram  imaging  has  been  simulated  through  rigorous  numerical 
algorithm,  finite  element  analysis  (FEA)  method 

3.  Two  types  of  mammogram  registration  model  have  been  developed.  MRI  guided  registration  model 
has  been  designed  for  cases  with  concurrent  mammograms  and  MR  images,  and  generic  registration 
model  has  been  developed  for  cases  with  only  mammograms 

4.  Datasets  have  been  constructed  for  this  project.  The  datasets  includes  dataset  of  breast  phantom 
mammograms  and  MR  images,  dataset  of  clinical  concurrent  mammograms  with  abnormal  cases  and 
control  cases,  dataset  of  temporal  mammograms  of  patients  containing  abnormal  and  control  cases; 
through  help  of  experienced  radiologists,  ground  truth  files  of  the  abnormal  cases  in  the  datasets  have 
been  constructed 

5.  Strict  evaluation  methods  to  test  performance  of  the  developed  registration  methods 

6.  The  developed  registration  methods  have  been  applied  on  the  concurrent  mammogram  datasets 

7.  The  developed  registration  methods  have  been  applied  on  the  temporal  mammogram  datasets 

8.  Performance  of  the  developed  mammogram  registration  methods  in  this  project  have  been  tested 
using  the  proposed  evaluation  methods 

4.  Reportable  Outcomes 


Manuscripts: 

1.  Qiu  Y,  Sun  XJ,  Monohar  V.,  Goldgof  D,  “Towards  Registration  of  Temporal  Mammograms  by 
Finite  Element  Simulation  of  MR  Breast  Volumes”,  Proceedings  of  SPIE  Medical  Imaging  2008 

2.  Qiu  Y,  Monohar  V.,  Korzhova  V.,  Sun  XJ,  Goldgof  D,,  “Two-View  Mammography  Registration 
using  3D  Finite  Element  Model  of  the  Breast”,  Submitted  to  Journal  of  Medical  Image  Analysis, 
2008 
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Presentations: 

1.  Goldgof  D,  Sun,  XJ,  “Accurate  3D  Modeling  of  Breast  Deformation  for  Temporal  Mammogram 
Registration”,  Era  of  Hope  2008  Meeting,  Baltimore,  USA,  June,  2008 

Grant  Applications: 

1.  BC084050,  DoD  Breast  Cancer  Idea  Award  application,  Principal  Investigator:  Dmitry  Goldgof, 

Title:  “Improve  breast  cancer  detection  through  registration  of  multiple  mammograms  by  accurate 
3D  modeling  of  breast  deformation”,  duration:  07/01/2009  -  06/30/2012,  total  amount; 

5.  Conclusion 

Mathematical  model  of  breast  deformation  has  been  developed  and  breast  compression  during 
mammographic  imaging  has  been  numerically  simulated.  Two  types  of  mammogram  registration  methods: 
magnetic  resonance  imaging  (MRI)  guided  registration  for  cases  with  concurrent  MR  images,  and  generic 
registration  for  cases  with  only  mammograms.  Datasets  have  been  constructed  to  validate  the  developed 
registrations  methods.  Mammogram  registrations  have  been  performed  on  concurrent  mammograms  using 
MRI  guided  and  generic  registration  method,  respectively.  Also  both  registration  methods  have  been 
applied  onto  temporal  mammograms. 

Promising  evaluation  results  on  mammogram  datasets  demonstrate  feasibility  of  the  developed 
mammogram  registration  methods.  The  results  also  indicate  that  the  developed  registration  methods 
would  be  very  helpful  in  mammogram  interpretation  for  improving  accuracy  of  detection  and  diagnosis  of 
breast  cancer. 
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ABSTRACT 

Performing  regular  mammographic  screening  and  comparing  corresponding  mammograms  taken  from  multiple 
views  or  at  different  times  are  necessary  for  early  detection  and  treatment  evaluation  of  breast  cancer,  which  is 
key  to  successful  treatment.  However,  mammograms  taken  at  different  times  are  often  obtained  under  different 
compression,  orientation,  or  body  position.  A  temporal  pair  of  mammograms  may  vary  significantly  due  to  the 
spatial  disparities  caused  by  the  variety  in  acquisition  environments,  including  3D  position  of  the  breast,  the 
amount  of  pressure  applied,  etc.  Such  disparities  can  be  corrected  through  the  process  of  temporal  registration. 
We  propose  to  use  a  3D  finite  element  model  for  temporal  registration  of  digital  mammography.  In  this  paper, 
we  apply  patient  specific  3D  breast  model  constructed  from  MRI  data  of  the  patient,  for  cases  where  lesions  are 
detectable  in  multiple  mammographic  views  across  time.  The  3D  location  of  the  lesion  in  the  breast  model  is 
computed  through  a  breast  deformation  simulation  step  presented  in  our  earlier  work.  Lesion  correspondence 
is  established  by  using  a  nearest  neighbor  approach  in  the  uncompressed  breast  volume.  Our  experiments  show 
that  the  use  of  a  3D  finite  element  model  for  simulating  and  analyzing  breast  deformation  contributes  to  good 
accuracy  when  matching  suspicious  regions  in  temporal  mammograms. 

Keywords:  mammography,  temporal  registration,  and  3D  finite  element  modeling. 

1.  INTRODUCTION 

Mammography  is  the  most  commonly  used  modality  for  early  breast  cancer  detection.  Mammograms  taken  in 
different  sessions  often  differ  in  tissue  configurations,  compression,  orientation,  or  body  position.  It  is  important 
to  match  features  detected  in  multiple  mammographic  views  across  time  to  diagnose  and  measure  treatment 
progress.  Hence,  some  form  of  spatial  non-rigid  transformation  of  image  data  is  required  so  that  the  tissues  are 
represented  in  an  equivalent  configuration.  However,  the  lack  of  3D  structural  knowledge  and  the  inhomogeneous 
nature  of  the  soft  tissues  within  the  breast  and  its  inherent  non-rigid  behavior  pose  a  challenge  to  the  registration 
procedure  between  temporal  mammograms.  Therefore,  a  3D  model  that  can  provide  accurate  information  about 
breast’s  3D  geometry  as  well  as  its  non-rigid  nature  during  deformation  is  strongly  desired.  In  this  paper  we 
discuss  a  temporal  registration  method  based  on  a  3D  finite  element  model  as  a  tool  for  simulating  and  analyzing 
breast  deformation  to  improve  the  accuracy  of  matching  in  temporal  mammograms. 

The  need  for  registration  across  the  time  dimension  can  be  realized  in  other  imaging  modalities  as  well  and 
there  are  approaches  developed  to  solve  this  problem.  Zana  et  al.1  presented  an  algorithm  for  temporal  and 
multimodal  registration  of  retinal  images  based  on  point  correspondences.  The  algorithm  was  applied  to  the 
registration  of  fluorescein  images.  The  vascular  tree  is  first  detected  in  each  type  of  images  and  bifurcation 
points  are  labeled  with  surrounding  vessel  orientations.  An  angle-based  invariant  is  computed  in  order  to  give  a 
probability  of  match  for  two  points.  A  Bayesian  Hough  transform  is  then  used  to  sort  the  transformations  with 
their  respective  likelihoods.  An  affine  estimate  is  computed  for  the  most  likely  transformations. 

Temporal  registration  has  also  been  used  in  breast  cancer  research  to  improve  the  sensitivity  of  mammograms. 
Marias  et  al.2  used  a  3- stage  mammogram  registration  method  that  involved:  (1)  boundary  registration  through 
the  detection  of  a  set  of  points  on  the  breast  boundary,  (2)  calculation  of  internal  correspondences  using  a  wavelet- 
based  multiscale  analysis,  and  (3)  calculation  of  the  transformation  using  both  the  boundary  and  the  internal 
points  in  a  thin-plate  spline  approximation  scheme.  It  would  be  desirable  if  breast  deformation  information  could 
also  be  used  in  the  computational  process  to  further  increase  the  registration  accuracy. 


We  propose  a  Finite  Element  Method  (FEM)  based  strategy  for  correspondence  identification  between  image 
features  identified  in  two- view  mammography  taken  at  different  times.  The  rest  of  the  paper  is  organized  in 
the  following  manner.  In  Section  2,  we  present  the  computational  methods  we  used  for  model  construction 
and  compression  simulation,  followed  by  a  description  of  the  registration  step.  In  Section  3,  we  present  the 
experimental  results  for  our  registration  algorithm  for  the  case  when  the  lesion  is  visible  in  both  views  in  the 
temporal  pair.  In  Section  4,  we  summarize  the  advantages  of  the  proposed  method  and  present  the  scope  for 
future  work  where  a  lesion  prediction  algorithm  can  be  applied  in  cases  when  the  lesion  is  not  detected  in  one 
or  more  of  the  mammographic  views  in  the  temporal  sequence. 


2.  COMPUTATIONAL  METHODS 

2.1  Breast  Finite  Element  Model  Construction  and  Deformation  Simulation 

The  finite  element  method  is  a  technique  for  modeling  deformable  objects.  The  approach  is  based  on  the 
underlying  geometry  and  the  material  properties  of  the  object.  Using  a  system  of  partial  differential  equations  to 
predict  the  movement  of  each  node,  shape  analysis  of  the  constitutive  elements  in  each  state,  material  properties 
of  the  object  under  consideration,  and  a  set  of  border  conditions  to  ensure  the  convergence  of  the  solution,  FEM 
can  predict  with  high  accuracy  the  final  state  of  the  object,  or  any  intermediate  states. 3-6 

The  basic  steps  of  the  FEM  approach  involving  object  deformations  are  the  following:  (1)  Derive  an  equilib¬ 
rium  equation  for  the  continuum  with  given  material  properties.  (2)  Select  the  appropriate  finite  elements  and 
corresponding  interpolation  functions  (also  called  shape  functions )  for  the  problem.  (3)  Subdivide  the  object 
into  the  elements  (called  meshing).  (4)  Obtain  the  stiffness  matrices  for  each  element.  (5)  Assemble  the  global 
stiffness  matrix  using  the  element  stiffness  matrices.  (6)  Impose  the  given  boundary  conditions.  (7)  Solve  the 
system  of  equations  for  the  vector  of  unknowns. 

The  female  breast  is  essentially  composed  of  four  structures:  lobules  or  glands,  milk  ducts,  fat,  and  connective 
tissue.  Most  biologic  tissues  have  both  a  viscous  and  an  elastic  response  to  external  deformations.  Since  we 
are  interested  only  in  slow  deformations,  the  response  of  the  tissue  can  be  considered  entirely  due  to  elastic 
forces.  As  a  first  approximation,  all  of  the  tissues  in  the  breast  were  assumed  to  be  isotropic,  homogenous,  and 
incompressible  with  nonlinear  elastic  properties  for  large  deformations.7-10 

We  generated  the  3D  breast  shape  model  for  a  patient  using  the  MRI  data.  The  model  was  discretized  using 
an  unstructured  tetrahedral  element  because  of  its  computational  efficiency  and  flexibility  in  handling  complex 
shapes.  10-node  tetrahedral  element  was  used  to  increase  modeling  accuracy  for  large  breast  deformation.  The 
dynamics  of  the  elastic  body  is  governed  by  the  following  system  of  partial  differential  equations: 


Pof#  =  (A  +  p) 

„  d2w  _  /\  i  ,.\  (  d2u  i  d2v  i  d2w\ 

PO  dt2  “lAt/iJ  ydxdz  ^  dydz  ^  3z 2  ) 
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+  fz, 
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where,  (u,  v,  w)  is  the  3D  displacement  vector,  fi  is  the  force  field,  and  p  and  A  are  the  Lame  constants  computed 
from  Young’s  modulus  and  Poisson’s  ratio. 

In  our  earlier  work,11  we  presented  the  details  of  our  method  for  breast  deformation  simulation  on  MR  images 
using  patient  specific  models.  Our  breast  model  construction  method  involved  several  steps  including  image 
extraction,  building  geometry  for  3D  model,  material  property  assignment,  and  boundary  condition  adjustment. 
Breast  deformation  was  approximated  by  an  incremental  stepwise  simulation  in  order  to  accurately  simulate  the 
dynamic  contact  problem  when  force  is  applied  through  the  plates  to  compress  the  breast.  Breast  deformation  in 
each  step  was  described  by  a  static  equilibrium  equation  and  the  Dirichlet  condition  (displacement)  was  specified 
on  the  plates.  The  detailed  breast  deformation  was  modeled  efficiently  and  accurately  by  adopting  an  adaptive 
re- meshing  strategy  that  assigned  smaller  sized  elements  in  the  region  of  interest.  More  details  of  the  method 
can  be  found  in.9  Figure  1  depicts  a  snapshot  of  the  breast  deformation  simulation  process. 

In  this  paper,  we  extend  our  method  on  breast  deformation  simulation  to  temporal  registration.  The  difference 
for  the  breast  compression  simulation  in  temporal  registration  is  that  the  compression  rate  which  was  recorded 


(a)  CC  view  compression  (white:  model  before  com-  (b)  MLO  view  compression  (white:  model  before  com¬ 
pression;  blue:  model  after  compression)  pression;  blue:  model  after  compression) 

Figure  1.  Model  CC  and  MLO  view  compression  simulation. 


in  all  of  the  views  in  different  temporal  pairs  can  be  applied  to  one  finite  element  model  of  the  patient.  Through 
the  simulations  of  breast  compression  and  relaxation,  the  3D  locations  for  the  lesion  in  the  mammograms  were 
linked  and  the  correspondence  was  established.  Figure  2  shows  a  sample  pair  of  mammograms  taken  in  two 
different  sessions  and  the  finite  element  model  of  the  breast  associated  with  the  patient. 


(a)  Mammogram  taken  at  (b)  Mammogram  taken  at  (c)  Model  constructed  using 
Time-1  (CC  View)  Time-2  (CC  View)  MRI  taken  at  Time-1 

Figure  2.  Sample  patient  mammograms  taken  at  two  different  times  and  the  associated  patient  breast  finite  element 
model. 

2.2  Registration  Procedure  for  Temporal  Mammograms 

In  this  section,  we  describe  our  registration  method  in  cases  where  a  lesion  is  visible  in  both  views  in  each 
temporal  pair.  We  call  the  pair  of  views  (CC  and  MLO)  taken  at  one  time  instant  as  temporal  pairs  TP1  and 
TP2. 

First,  we  performed  the  compression  simulation  for  the  temporal  pair  TP1  according  to  the  compression 
rate  recorded  in  the  mammograms.  The  3D  location  of  the  lesion  using  these  two  views  is  predicted  using  back 
projection  onto  the  model.  Complete  details  of  this  process  can  be  found  in  our  earlier  work.9  After  we  completed 
the  same  procedure  for  the  temporal  pair  TP2,  we  compared  both  the  predicted  3D  locations  from  TP1  and  TP2 


in  the  finite  element  model  and  used  the  Euclidean  distance  as  the  measure  of  correspondence  error.  Figure  3 
shows  an  illustration  of  the  registration  procedure. 


TP2  Lesion  Prediction 

Figure  3.  Illustration  for  the  registration  procedure. 


A  summary  of  the  steps  undertaken  is  as  follows: 

1.  For  the  temporal  case  TP1, 

(a)  Compression  simulation  on  the  finite  element  model  of  the  patient  is  performed  according  to  the 
compression  rate  recorded  in  the  mammograms. 

(b)  After  aligning  with  the  corresponding  view  based  on  the  direction  recorded  in  the  mammograms, 
the  locations  of  all  the  features  (lesions)  identified  in  one  view  (CC)  are  back  projected  onto  the 
compressed  breast  model  as  lines.  Back  projection  is  accomplished  through  a  simple  ray  tracing 
method  with  known  camera  configuration. 

(c)  The  model  is  adaptively  re- meshed  in  the  regions  adjacent  to  the  straight  lines  generated  in  step  (b) 
and  the  elements  through  which  the  straight  lines  pass  are  labeled. 

(d)  The  compressed  breast  is  restored  to  its  natural  shape  and  a  3D  curve  is  obtained.  As  a  result,  each 
feature  location  in  one  view  will  have  a  corresponding  3D  curve  in  the  uncompressed  model. 

(e)  Steps  (b)  -  (d)  are  repeated  for  each  feature  found  in  the  second  view  (MLO). 

(f)  A  feature  in  CC  view  is  paired  with  all  features  in  MLO  view  and  the  corresponding  distance  between 
their  3D  curves  is  measured  (the  distance  between  two  3D  curves  was  defined  as  the  minimum  distance 
between  two  nodes  on  these  curves). 


(g)  All  of  the  feature  pairs  are  ranked  based  on  the  distances  between  their  3D  curves  in  the  uncompressed 
model.  The  pair  with  the  minimum  distance  is  considered  as  a  match.  In  other  words,  they  are  more 
likely  related  to  the  same  breast  lesion. 

(h)  The  intersection  of  the  corresponding  3D  curves  provides  the  reconstructed  3D  position  of  this  feature. 

(i)  Step  (f)  -  (h)  is  repeated  for  each  feature  in  CC  view. 

2.  The  registration  procedure  (1)  is  performed  on  the  temporal  pair  TP2  and  the  3D  coordinates  of  each 
feature  within  the  reconstructed  natural  breast  volume  is  estimated. 

3.  The  Euclidean  distances  between  all  pairs  of  predicted  locations  along  the  temporal  mammograms  are 
calculated.  Pairs  that  have  the  minimum  distance  in  the  reconstructed  breast  shape  are  matched. 

Figure  4  shows  an  overview  of  the  system  flow  of  the  proposed  method  including  all  of  the  pre-processing 
and  the  post-processing  procedures  for  breast  deformation  simulation. 


Figure  4.  System  flow  for  the  proposed  approach. 


3.  EXPERIMENTAL  RESULTS 

In  this  section,  we  present  results  for  our  registration  procedure  for  cases  where  lesion  is  visible  in  both  views 
in  the  temporal  pairs.  MR  volumes  of  patient  breast  were  used  for  model  construction  and  the  corresponding 
mammograms  were  selected  for  compression  simulation  experiments. 


For  constructing  the  individual  models,  patient  breasts  were  scanned  with  an  MRI  scanner  at  the  Lifetime 
Screening  Center,  H.  Lee  Moffitt  Cancer  Center  &  Research  Institute,  University  of  South  Florida.  The  voxel 
size  for  the  TP1  MRI  is  1.41  x  1.41  x  2.50  mm.  Four  sets  of  patients  MRIs  were  provided  for  the  patients  with 
corresponding  2- view  mammographic  pairs  at  two  different  times.  Mammograms  were  digitized  at  a  resolution  of 
75  micron  and  12  bits  per  pixel,  containing  mass  or  calcification  (three  instances  of  calcification  and  one  instance 
of  mass).  Patient  age  group  ranged  from  30  to  70  years  with  a  median  of  45  years.  The  images  were  collected  over 
a  period  of  time  from  daily  clinical  investigations.  More  data  are  still  being  provided  by  the  Lifetime  Screening 
Center  for  various  validation  purposes.  The  center  of  a  lesion  is  determined  as  the  center  of  a  circle  (or  an  ellipse) 
manually  fitted  by  a  physician  or  automatically  identified  by  a  Computer  Aided  Detection  (CAD)  system  based 
on  the  visible  parts  of  the  lesion  in  the  patient  datasets. 

We  analyzed  mammograms  by  examining  temporal  sequences  of  images.  The  task  of  comparing  mammograms 
is  difficult  because  there  are  many  factors  which  may  cause  changes  in  image  appearance,  e.g.,  choice  of  image 
acquisition  parameters,  positioning  and  compression  of  the  breast,  image  display  parameters,  and  changes  in 
breast  anatomy.12, 13  Differences  caused  by  changes  in  breast  positioning  and  compression  are  more  complex 
and  more  difficult  to  correct  because  mammograms  are  projections  through  the  deformed  breast.  After  model 
compression  and  relaxation,  as  described  in  Section  2,  the  average  correspondence  error  in  the  four  patient 
datasets  using  the  measure  of  Euclidean  distance  between  the  predicted  lesion  positions  in  TP1  and  TP2  was 
1.6±0.4mm.  One  can  note  that  this  error  is  acceptable  when  compared  to  the  average  lesion  size  (10mm  -  40mm). 
Since  there  was  only  one  suspicious  region  per  case,  this  validates  the  accuracy  of  the  temporal  registration 
algorithm. 


4.  CONCLUSIONS  AND  FUTURE  WORK 

We  presented  a  deformable  3D  finite  element  model  based  method  to  improve  the  accuracy  of  lesion  registration 
between  temporal  mammographic  views.  The  advantage  of  using  a  3D  finite  element  model  is  that  non-rigid 
breast  deformation  can  be  computed  accurately,  which  is  lacking  in  2D  registration  methods.  We  used  a  step¬ 
wise  incremental  approach  to  simulate  the  plate’s  motion,  which  enables  us  to  model  large  breast  deformation 
through  a  series  of  static  equilibrium  calculation.  We  also  employed  an  adaptive  meshing  technique  to  reduce 
the  computational  cost  while  maintaining  the  required  computational  accuracy.  The  use  of  MRIs  of  the  patient 
to  build  a  finite  element  model  further  ensured  the  registration  quality. 

A  dataset  containing  four  patient  data  (2- view  mammographic  pairs  at  two  different  times  and  MR  volume) 
was  built  and  tested  with  patient  specific  models  constructed  using  the  finite  element  method.  Our  experimental 
results  showed  that  the  proposed  method  could  confirm  lesion  correspondence  in  temporal  pairs  of  mammograms 
with  reasonable  accuracy.  The  proposed  modeling  approach  holds  great  promise  in  both  early  breast  cancer 
diagnosis  and  the  subsequent  treatment  evaluation. 

For  future  study,  with  the  validation  experiments  discussed  in  this  paper,  we  plan  to  extend  the  proposed 
approach  for  cases  when  lesion  is  not  detected  in  one  or  more  mammograms  in  the  temporal  sequence.  The 
3D  location  of  the  lesion  in  the  finite  element  model  can  be  projected  onto  the  mammogram  as  a  line.  This 
potential  region  for  the  lesion’s  location  can  be  used  by  the  physician  for  better  diagnosis.  Further,  a  local 
image  enhancement  method  could  be  applied  on  this  predicted  lesion  region  to  further  improve  the  sensitivity  of 
mammograms. 
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Abstract 

This  paper  presents  a  novel  method  for  modeling  breast  deformation  and  using  it  as 
an  effective  tool  for  two-view  mammography  registration.  The  approach  is  based  on 
automatic  three-dimensional  (3D)  registration  between  mammograms  and  Magnetic 
Resonance  (MR)  images.  A  3D  finite  element  method  (FEM)  is  used  to  model  and 
simulate  breast  deformation  during  mammography,  in  which  two  deformation  models 
are  built  depending  on  availability  of  MR  images  of  the  breast.  For  cases  that  have 
both  MR  data  and  mammograms,  patient  specific  breast  deformation  model  is  built, 
while  for  cases  that  only  have  mammograms,  a  generic  breast  deformation  model  is 
developed.  A  3D  registration  algorithm  aligns  breast  lesions  identified  in  two  different 
views  by  applying  the  deformation  model  and  is  subsequently  used  in  finding  precise 
locations  of  breast  lesions  within  the  natural  breast  volume.  An  evaluation  of  3D  reg¬ 
istration  using  the  patient  specific  deformation  model  on  22  clinical  data  sets  resulted 
in  an  accurate  localization  with  a  mean  error  of  about  3.0  mm  and  2.2  mm  for  lesion 
position  prediction  in  mammograms  and  MR  images,  respectively.  The  registration 
algorithm  using  generic  breast  models  with  47  clinical  data  sets  resulted  in  a  mean  2D 
localization  error  of  about  4.7  mm  in  predicting  the  lesion  position  in  mammograms. 

Key  words:  Breast  deformation,  registration,  finite  element  model,  mammogram, 
Magnetic  Resonance  Imaging. 


1.  Introduction 

Currently  mammography  (X-ray  imaging  of  the  breast)  is  the  most  commonly  used 
imaging  modality  for  breast  cancer  screening  and  diagnosis.  Many  studies  suggest  that 
screening  using  two-view  mammography  is  more  effective  than  a  one-view  system, 
especially  for  cancers  of  small  size  [1,  2,  3].  However,  in  current  clinical  settings, 
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there  is  still  a  high  amount  of  uncertainty  involved  in  the  readings  of  radiologist  for 
two- view  mammograms.  In  interpreting  mammograms,  three  problems  are  commonly 
encountered:  (1)  How  to  relate  suspicious  findings  (mass  and  calcification)  from  two 
views  to  determine  the  existence  of  cancer?  (2)  If  a  suspicious  area  is  found  only  in  one 
view,  what  is  its  possible  position  in  another  view?  (3)  When  finding  breast  lesion(s) 
in  mammograms,  how  to  locate  its  exact  position  in  the  natural  breast  volume? 

The  fact  that  there  is  mutual  disagreement  between  the  readings  recorded  by  differ¬ 
ent  radiologists  further  intensifies  these  problems.  To  improve  the  quality  of  interpre¬ 
tation,  a  3D  model  will  be  helpful  in  providing  accurate  information  about  breast’s  3D 
geometry  as  well  as  its  deformation. 

Three  types  of  models  have  been  used  in  two- view  mammography  - 

Statistical  Models:  Masses  were  detected  in  digital  mammograms  using  an  ip- 
silateral  multi-view  CAD  system  that  exploited  correlative  information  of  suspicious 
lesions  between  mammograms  of  the  same  breast  [4].  In  [5],  a  genetic  algorithm-based 
method  to  classify  and  detect  microcalcification  clusters  in  digitial  mammograms  was 
proposed.  After  transforming  the  input  images  into  an  appropriate  feature  domain, 
chromosomes  were  constructed  to  populate  the  initial  generation.  Further  features  were 
extracted  to  allow  the  proposed  GA  to  search  for  optimized  classification  and  detection 
of  microcalcification  clusters.  Sahiner  et  al.  [6]  used  a  classifier  to  analyze  the  similar¬ 
ity  between  feature  pairs  in  cranio-caudal  (CC)  and  mediolateral  oblique  (MLO)  views. 
Scores  from  the  classifier  were  then  used  to  improve  single-view  detection.  Using  the 
datasets  from  the  University  of  Michigan  and  the  University  of  South  Florida,  they  re¬ 
ported  a  0.58  false  positive  rate  with  two  views  and  a  0.73  false  positive  rate  with  a 
single  view.  In  [7],  a  preprocessing  step  that  improved  the  image  contrast  based  on 
local  statistical  measures  of  mammograms  was  proposed.  After  enhancement,  regions 
were  segmented  using  multiple  level  thresholding  and  a  set  of  features  from  each  seg¬ 
mented  region  was  computed.  A  region  ranking  system  identified  sections  that  most 
likely  represented  abnormalities  based  on  the  computed  features.  Suliga  et  al.  [8]  pre¬ 
sented  a  Markov  Random  Field  based  technique  that  labeled  the  image  pixels  based 
on  the  description  of  their  statistical  and  contextual  information.  The  method  also  ex¬ 
panded  the  analysis  by  a  description  of  the  spatial  dependence  between  pixels  and  their 
labels,  thus  leading  to  a  reduction  in  the  inhomogeneity  of  output.  Though  the  above 
approaches  are  characterized  by  their  simplicity  which  allow  them  to  be  run  on  vir¬ 
tually  any  kind  of  platform,  they  use  very  limited  information  (such  as  the  distance 
between  features  and  nipples  in  [6]),  which  limits  the  reliability  of  these  methods. 

Geometrical  Models:  Kita  et  al.  [2]  used  a  simplified  geometrical  model  to  com¬ 
pute  breast  deformation  and  then  established  feature  correspondences.  In  [3],  a  more 
sophisticated  model  was  used  to  reconstruct  micro-calcification  clusters.  A  modified 
projective  grid  space  (MPGS)  scheme  was  proposed  to  reconstruct  3D  microcalcifi¬ 
cations  in  a  unique  space  defined  by  corresponding  points  and  the  epipoles  retrieved 
from  the  fundamental  matrix  of  the  CC  and  MLO  views  [9].  In  order  to  account  for 
breast  deformation,  a  new  technique  based  on  the  concept  of  bundle  adjustment  was 
proposed  to  rectify  the  3D  locations  of  reconstructed  microcalcifications  in  the  real 
patient  body  model  (constructed  using  the  MPGS  scheme  with  Iterative  Closest  Point 
algorithm).  Geometrical  models  have  a  drawback  that  many  assumptions  have  to  be 
made  to  idealize  the  breast  deformation,  which  may  not  be  valid  for  a  highly  deformed 
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breast. 

Biomechanical  Models:  Non-rigid  motion  has  been  extensively  studied  in  the 
computer  vision  community.  Huang  [10]  classified  non-rigid  motion  into  different 
categories.  Aggarwal  et  al.  [11]  provided  a  comprehensive  survey  on  various  modeling 
approaches,  especially  those  based  on  physical  models.  For  example,  biomechanical 
models  were  used  to  study  breast  biopsy  and  MRI/X-ray  data  fusion  [12,  1].  A  fully 
automatic  method  based  on  a  deformable  model  was  presented  to  detect  vessels  in 
mammograms  and  was  shown  to  have  an  excellent  performance  level  in  terms  of  ac¬ 
curacy,  sensitivity,  and  specificity  [13].  Wildes  et  al.  [14]  proposed  a  simple  physical 
breast  model  for  registration  that  viewed  the  breast  as  a  set  of  tissue  compartments 
contained  within  an  outer  skin.  Highnam  et  al.  [15]  used  a  compression  model  to  de¬ 
termine  correspondence  between  the  CC  and  MLO  views.  Bakic  et  al.  [16]  proposed 
a  non-rigid  breast  model  for  task-driven  mammogram  segmentation.  Biomechanical 
models  have  the  capability  of  handling  irregular  shapes  and  predicting  breast  defor¬ 
mation  more  accurately.  A  comprehensive  review  of  deformable  modeling  techniques 
and  their  applications  in  medical  image  analysis  can  be  found  in  [17]  and  [18,  19], 
respectively. 

Schnabel  et  al.  proposed  a  method  [20]  for  validation  of  non-rigid  medical  image 
registration  for  breast  MRIs  using  biomechanical  models.  The  method  is  based  on  the 
simulation  of  biomechanical  tissue  deformations  using  finite  element  methods  (FEM). 
It  involves  a  non-rigid  image  registration  step  based  on  free-form  deformations  using 
B -splines.  The  voxel  similarity  measure  was  computed  using  normalized  mutual  in¬ 
formation.  The  efficacy  of  the  method  was  demonstrated  on  contrast  enhancement  of 
magnetic  resonance  mammographic  image  pairs  as  a  prototype  application. 

Different  breast-imaging  modalities  bring  complementary  information  that  can  help 
in  establishing  a  concrete  diagnosis.  Researchers  have  been  working  on  inter-modality 
breast  image  registration  and  on  the  design  of  co-registered  multi-modality  breast  imag¬ 
ing  system.  To  combine  multimodal  information,  Coman  et  al.  [21]  proposed  a  method 
for  non-rigid  co-registration  of  PET  and  MR  breast  images.  The  model  allows  estima¬ 
tion  of  the  inter-modal  breast  deformation  to  determine  the  location  within  the  breast. 
The  FEM  “loads”  were  taken  as  the  observed  inter-modal  displacements  of  several 
fiducial  skin  markers  placed  on  the  breast  and  visible  in  PET  and  MRI.  The  analogy 
between  orthogonal  components  of  the  displacement  field  and  the  temperature  differ¬ 
ences  in  a  steady-state  heat  transfer  (SSHT)  in  solids  was  adopted.  The  model  assumed 
that  there  were  no  stress  induced  deformations  and  used  external  fiducial  markers  for 
the  multimodal  data  (PET  and  MRI).  The  method  achieved  less  than  5  mm  error  in 
registration. 

In  clinical  practice,  mammograms  are  more  commonly  used  than  PET.  Registration 
of  an  X-ray  mammogram  and  an  MR  volume  is  a  2D/3D  problem.  One  can  construct 
2D  projection  images  from  the  volume  and  apply  a  2D/2D  registration  approach  to  ob¬ 
tain  an  optimal  match.  This  strategy  can  be  found  in  the  existing  literature  [22,  23,  24]. 
Behrenbruch  et  al.  [22]  proposed  a  method  in  which  an  MR  breast  volume  was  pro¬ 
jected  as  an  image  representing  uncompressed  tissue.  The  method  projected  the  actual 
voxel  enhancement  characteristics  via  a  pharmaco-kinetic  model  where,  low  enhance¬ 
ment  tissues  were  given  a  low  intensity  level  in  the  “pseudo  X-ray”  projection.  On 
the  other  hand,  high-enhancement  voxels  were  given  a  correspondingly  high  intensity 
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value.  It  would  be  desirable  if  the  model  deformation  could  be  combined  before  pro¬ 
jection  to  further  improve  the  accuracy  of  registration. 

To  take  the  deformation  into  consideration,  Bakic  et  al.  [25]  proposed  a  system¬ 
atic  model  approach  to  evaluate  sensitivity  of  registration  methods  to  various  types  of 
changes  in  mammograms  using  synthetic  breast  images  with  known  deformations.  The 
breast  model  had  a  shape  defined  by  an  ellipsoidal  approximation  of  the  breast  outline 
and  an  ellipsoidal  approximation  of  a  border  between  internal  regions  with  predom¬ 
inantly  adipose  tissue  and  those  with  predominantly  fibroglandular  tissue.  Since  the 
synthetic  model  used  an  ellipsoidal  approximation,  it  cannot  fully  describe  the  shape 
of  patient  breast  which  ultimately  affects  the  registration  accuracy.  We  refer  the  reader 
to  [26]  for  an  overview  of  both  intra-modality  and  inter-modality  breast  image  regis¬ 
tration  techniques. 

In  our  earlier  work  [27,  28,  29],  we  presented  the  main  idea  of  utilizing  a  finite 
element  model  of  the  breast  constructed  from  breast  MR  images  to  model  breast  com¬ 
pression  during  mammographic  imaging  and  utilizing  the  simulation  for  automatic  reg¬ 
istration  of  X-ray  mammograms  and  MR  images.  The  approach  has  several  advantages: 
(1)  The  use  of  MR  images  allows  us  to  set  up  precise  breast  deformation  model;  (2) 
An  incremental  contact  simulation  scheme  gives  more  accurate  descriptions  of  breast 
deformation  ensuring  registration  quality;  (3)  An  adaptive  meshing  method  enables  us 
to  strike  a  balance  between  computational  efficiency  and  modeling  accuracy. 

Ruiter  et  al.  [30]  used  a  similar  approach  for  automatic  localization  of  lesions  which 
are  visible  either  in  the  mammograms  or  in  the  MR  image.  By  using  a  3D  finite  element 
model  of  the  deformable  behavior  of  breast,  they  account  for  the  huge  deformation 
during  mammography  and  the  3D  effects  during  deformation.  Results  on  six  clinical 
cases  presented  an  average  lesion  localization  error  of  4.3  ±1.0  mm  (in  mammograms) 
and  3.9  ±1.7  mm  (in  MR  images). 

In  a  very  recent  study  [31],  a  method  for  simulating  mechanical  compression  of 
volumetric  CT  breast  data  was  presented.  The  method  consists  of  the  segmentation 
and  classification  of  the  volumetric  data  into  various  material  types,  followed  by  a  finite 
element  algorithm  for  compression  simulation.  The  approach  was  unique  in  the  sense 
it  permitted  the  assignment  of  individual  physical  mechanical  properties  to  each  node 
based  on  the  local  tissue  configuration.  Qualitative  results  demonstrated  the  possibility 
of  compression  simulation  for  a  variety  of  objects  (including  breast)  using  the  method. 

In  this  paper,  we  put  forward  the  finite  element  modeling  based  approach  intro¬ 
duced  in  [27,  28,  29]  as  an  effective  way  to  combine  information  present  in  MRIs 
and  mammograms.  When  compared  to  [30],  this  paper  makes  the  following  sub¬ 
stantial  contributions:  (1)  We  propose  the  use  of  a  generic  breast  modeling  method 
for  cases  whose  MR  images  are  not  available.  The  method  presented  in  this  paper 
adapts  a  generic  3D  breast  deformation  model  based  on  the  breast  mammograms  and 
the  recorded  compression  amount  and  direction;  (2)  For  accurate  simulation  of  plate 
compression  (a  dynamic  contact  problem),  we  utilze  an  incremental  stepwise  approach 
in  which  the  plate  motion  is  slow  enough  that  breast  deformation  in  each  step  can  be 
described  by  a  static  equilibrium  equation  and  more  importantly,  mesh  topology  will 
not  be  too  distorted  to  affect  displacement  prediction;  (3)  We  support  the  proposed 
approach  with  extensive  experimental  results. 
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2.  Finite  Element  Model  and  Breast  Deformation 


Finite  Element  Method  is  a  technique  for  modeling  deformable  objects.  It  has  been 
used  to  study  mechanical  behavior  of  many  human  organs  such  as  heart,  lung,  kidney, 
as  well  as  breast.  The  approach  is  based  on  the  underlying  geometry  and  the  material 
properties  of  the  object.  Using  a  system  of  partial  differential  equations  to  predict  the 
movement  of  each  node,  shape  analysis  of  the  constitutive  elements  in  each  state,  mate¬ 
rial  properties  of  the  object  under  consideration,  and  a  set  of  border  conditions  to  ensure 
the  convergence  of  the  solution,  FEM  can  predict  with  high  accuracy  the  final  state  of 
the  object,  or  any  intermediate  states  [20,  32,  12,  33].  There  are  many  commercial 
FEM  packages  the  allow  complex  simulations  of  deformation.  We  used  ANSYS  [34] 
for  numerically  solving  the  partial  differential  equations  of  elasticity  theory. 

The  female  breast  is  essentially  composed  of  four  structures:  lobules  or  glands, 
milk  ducts,  fat,  and  connective  tissue.  Most  biologic  tissues  have  both  a  viscous  and 
an  elastic  response  to  external  deformations.  Since  we  are  interested  only  in  slow 
deformations,  the  response  of  the  tissue  can  be  considered  entirely  due  to  elastic  forces. 

In  [35],  a  study  was  done  to  analyze  the  effect  of  tissue  density  on  cancer  detection. 
Results  showed  that  breast  density  is  an  important  factor  affecting  not  only  radiologist’s 
reading  but  also  CAD  performance.  The  study  concluded  that  in  order  to  improve  early 
detection  of  breast  cancer,  a  special  effort  should  be  directed  towards  accurate  model¬ 
ing  of  breast  tissues  in  a  CAD  system  design.  This  raises  an  important  issue  to  be  ad¬ 
dressed  for  accurate  deformation  simulation  -  the  need  to  know  the  material  property 
of  the  breast  tissues.  Due  to  the  difficulty  of  obtaining  “true”  material  property  values, 
it  is  common  that  soft  tissues  are  modeled  as  isotropic,  linear,  and  homogeneous.  In 
the  context  of  feature  registration,  using  an  isotropic  property  is  a  reasonable  choice, 
although  the  effect  of  the  Cooper’s  ligaments  on  the  overall  property  of  breast  tissue 
is  still  not  clear.  Due  to  the  large  breast  deformation  caused  by  compression,  more  at¬ 
tention  was  directed  to  the  non-linear  behavior  of  breast  tissues.  For  example,  Azar  et 
al.  [1]  used  an  exponential  function  to  approximate  the  non-linear  relationship  between 
stress  and  strain.  Similarly,  Samani  et  al.  [12]  used  a  polynomial  function  to  account 
for  the  non-linearity  of  breast  tissue  properties.  In  these  studies,  the  non-linearity  was 
considered  for  both  fat  and  glandular  tissues,  using  the  coefficients  obtained  from  ei¬ 
ther  a  tensile  test  or  the  tactile  imaging  results  [36].  In  the  study  by  Tanner  et  al.  [37], 
the  authors  indicate  that  the  impact  of  a  linear  assumption  on  breast  modeling  is  not  as 
significant  as  that  of  boundary  settings.  In  this  study,  as  a  first  approximation,  all  of 
the  tissues  in  the  breast  were  assumed  to  be  isotropic,  homogenous,  and  incompress¬ 
ible  with  a  linear  approximation  for  the  mechanical  behavior  of  breast  tissue  for  small 
deformation  [27,  28,  29].  Based  on  findings  in  the  literature  [12,  20,  37,  38,  39]  and 
further  model  calibration,  a  uniform  Young’s  modulus  of  15  kPa  and  a  Poisson’s  ratio 
of  0.495  were  used. 

Under  the  assumptions  that  breast  is  incompressible  with  isotropic  property,  breast 
deformation  can  be  described  by  the  following  set  of  partial  differential  equations: 
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where,  po  is  the  density,  (u,  v,  w)  is  the  3D  displacement  vector,  ( fx ,  fy,  fz)  is  the  force 
field,  and  /i  and  A  are  the  Lame  constants  computed  from  Young’s  modulus  (. E )  and 
Poisson’s  ratio  (v)  as  n  =  2(1^.t/)  and  A  =  (1+„^_2t/)  ■ 

The  FE  model  domain  was  discretized  using  unstructured  3D  tetrahedral  structural 
solid  element  because  of  its  computational  efficiency  and  flexibility  in  handling  com¬ 
plex  shapes.  Each  tetrahedral  element  was  composed  of  10  nodes  (each  side  has  an 
additional  node  in  the  middle  to  model  the  deformations  more  accurately).  The  ele¬ 
ments  exhibit  a  quadratic  behavior  for  interpolation  purposes.  The  resulting  meshed 
volume  is  presented  in  Figure  1.  This  was  the  finite  element  model  of  the  physical 
phantom  to  which  the  deformations  was  applied.  For  a  10-pixel  sample  interval  in  the 
original  image  slice,  52  slices  were  stacked  to  construct  the  volume. 


(a)  Digital  image  of  phantom  (b)  FE  Model  after  meshing 

Figure  1 :  Finite  Element  Model  of  the  Phantom. 


Table  1  shows  the  resulting  number  of  nodes  and  elements  in  the  finite  element 
model  of  the  phantom  for  varying  element  sizes.  In  the  further  experiments  described 
in  this  paper,  we  used  an  element  size  of  8  units  and  the  ANSYS  meshing  procedure 
resulted  in  13225  nodes  and  8744  elements. 


Element  Size 

Number  of  nodes 

Number  of  elements 

20 

2836 

1668 

15 

4010 

2424 

10 

7899 

5068 

8 

13225 

8744 

5 

52982 

36824 

Table  1 :  Number  of  Nodes  and  Elements 


During  X-ray  imaging,  force  is  applied  through  two  plates  that  move  towards  each 
other  to  compress  breast.  This  is  a  dynamic  contact  problem  that  must  be  simulated 
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numerically.  We  approximated  breast  deformation  during  compression  by  incremental 
stepwise  simulation.  The  underlying  assumption  is  that  the  motion  of  plate  is  slow 
enough  so  that  breast  deformation  in  each  step  can  be  described  by  a  static  equilibrium 
equation.  More  importantly,  the  mesh  topology  will  not  be  significantly  distorted  to  in¬ 
fluence  the  displacement  prediction.  Also,  such  an  iterative  approach  with  the  boundary 
(contact  surface)  being  constantly  updated  divides  the  large  deformation  compression 
into  a  number  of  much  smaller  displacement  steps.  Therefore,  the  small  deformation 
linearity  assumption  of  the  method  is  preserved. 

In  clinical  practice,  the  final  compression  magnitude  is  recorded,  but  the  force  ex¬ 
erted  on  plates  is  rarely  measured.  So,  we  specified  the  Dirichlet  condition  (displace¬ 
ment)  on  the  plates.  To  avoid  sliding  movement  between  plates  and  breast,  we  assumed 
that  once  in  contact  with  the  plates,  the  node  will  move  only  in  the  direction  of  com¬ 
pression. 

One  boundary  condition  that  appears  often  in  breast  modeling  is  that  nodes  at  the 
chest  wall  have  zero  displacement  vectors  since  the  breast  is  connected  through  the 
chest  wall  to  the  body.  To  constrain  the  FE  model  when  the  contact  surface  between 
the  breast  and  the  plates  is  modeled,  the  median  nodes  are  restricted  to  zero  displace¬ 
ment  vectors  along  the  z-axis  [37].  In  our  model  we  waived  these  boundary  conditions 
because  of  the  following  reasons:  (i)  During  compression,  the  breast  is  not  rigidly 
fixed  at  the  chest  wall  and  the  tissues  are  displaced  around  it;  (ii)  The  restriction  on  the 
degrees  of  freedom  for  the  median  nodes  is  an  artificial  condition  meant  only  for  con¬ 
vergence  purposes  rather  than  a  natural  one  since  one  cannot  predict  accurately  which 
tissue  inside  the  breast  will  have  zero  displacement  along  the  z-axis.  The  only  bound¬ 
ary  conditions  imposed  on  our  model  are  the  following:  (i)  All  nodes  in  the  first  two 
planes  at  the  base  of  the  breast  model  are  fixed.  This  provides  the  support  structure 
and  fixes  the  breast  model  in  space,  (ii)  The  contact  between  the  breast  and  the  plates 
is  a  rough  contact  without  any  sliding,  and  the  plates  are  restricted  to  move  only  along 
the  z-axis.  This  boundary  condition  is  a  natural  one  and  doesn’t  affect  the  convergence 
of  the  solution  to  Equation  (1).  Figure  2  depicts  a  snapshot  of  the  breast  deformation 
simulation  process. 


(a)  CC  View  (white:  model  before  (b)  ML  View  (white:  model  before 

compression;  blue:  model  after  compression;  blue:  model  after 

compression)  compression) 

Figure  2:  Model  CC  and  ML  View  Compression  Simulation. 
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3.  Algorithm  for  Two-View  Mammogram  Registration 

We  developed  two  models  to  simulate  breast  deformation.  One  was  a  patient- 
specific  model  which  was  used  when  both  mammograms  and  MR  images  were  avail¬ 
able  for  the  patient,  and  the  other  was  a  generic  model  which  was  used  when  only 
mammograms  were  available.  We  divide  our  approach  in  two  parts,  namely,  (1)  Breast 
Model  Construction;  (2)  Registration. 

3.1.  Breast  Model  Construction 

Patient  Specific  Models:  In  cases  when  the  patient  data  consisted  of  both  mamma- 
grams  and  MRIs,  we  used  MR  data  to  construct  the  3D  breast  model  of  the  patient  [27]. 

In  each  MRI  slice,  breast  was  segmented  and  the  2D  breast  contour  was  extracted 
using  standard  morphological  operators.  B- spline  smoothing  was  implemented  to  re¬ 
move  small  sharp  edges  that  might  have  been  generated  during  segmentation.  Then, 
the  3D  breast  shape  was  constructed  by  combining  all  the  2D  breast  contours  as  shown 
in  Figure  3.  For  more  details  about  the  model  construction,  we  refer  the  reader  to  [27]. 


(a)  MRI  Slices 


(c)  Non  Planar  Area  Fitted  on  Spline  (d)  Constructed  Natural  Breast  Volume 

Curves  (Coons  Patch) 

Figure  3:  Construction  of  Natural  Breast  Volume  using  MR  Images. 


8 


It  is  worth  noting  that  the  model  ignores  any  deformation  due  to  gravitational 
forces.  The  use  of  unloading  methods  [40]  that  specifically  address  the  gravity  loaded 
state  of  the  breast  will  likely  provide  more  reliable  diagnosis  and  accurate  lesion  local¬ 
ization. 

Generic  Model:  In  clinical  practice,  patient  MRIs  and  mammograms  are  not  al¬ 
ways  obtained  at  the  same  time  or  MRI  may  not  be  taken  at  all.  To  assist  the  physician 
for  better  localization  of  lesion  or  finding  correspondence,  we  propose  the  use  of  a 
generic  model  for  correspondence  recovery. 

The  proposed  algorithm  adjusts  a  generic  3D  FE  breast  model  based  on  patient 
mammograms  and  known  compression  amount  and  direction  recorded  in  mammo¬ 
grams.  The  initial  generic  model  was  selected  from  the  library  of  existing  patient 
MR  volumes  based  on  the  criterion  of  similar  proportion.  The  mammograms  were 
compared  to  the  generic  model  projections  to  obtain  several  scaling  factors.  The  three 
dimensional  scaling  factors  were  computed  using  the  ratio  of  patient  breast  and  pro¬ 
jection  in  x,  y,  and  z  directions.  The  following  equations  were  used  to  calculate  the 
scaling  factors. 


X  C  C  .M  odelProjecti 

Cj  _  YML-Xray 

“ y  ~  yml  -M  odelProjectr! 


ZcC.Xr 


ZcC.ModelProj  ection  ZMl  -M  odelProj  ect 


ZmL.Xt 


(2) 


where  Xcc ,  Ycc,  and  Zcc  describes  the  dimensions  of  the  patient  breast  or  the  model 
projection  in  the  x,  y,  and  z  directions  respectively. 

An  error  component,  E,  was  calculated  as, 


E  = 


_ A  Areacc _ 

Areacc  -M  odelProj  ect  i> 


+ 


_ AAreaM  l _ 

Ar  ect  m  L  _M  odelProj  ection 


) 


(3) 


When  E  was  above  a  pre-defined  threshold,  further  adjustment  for  better  boundary 
fitting  was  needed.  We  used  two  parameters  (say  S4  and  S5)  for  boundary  adjustment. 
64  was  calculated  based  on  the  maximum  deviation  ratio  in  ML  views  and  S 5  was 
calculated  based  on  the  maximum  deviation  ratio  in  CC  views  (Equation  (4)). 


54 

55 


_ Areacc  - X  ray _ 

Areacc -Model  Pro  j  ection  5 
_ AreaML-Xray _ 

AreaM  L-M  odelProj  ection 


(4) 


Maximum  deviation  ratio  is  calculated  based  on  the  overlapping  area  for  the  cor¬ 
responding  views.  For  each  contour  which  forms  the  generic  model,  the  points  on  the 
contour  was  adjusted  using  64  or  S5  (S4  for  ML  view  and  S5  for  CC  view).  A  new 
model  was  constructed  using  the  adjusted  contour  and  the  projection  error  based  on  the 
overlapping  area  was  calculated.  The  above  process  was  performed  with  several  initial 
generic  models  and  the  one  with  the  minimum  error  was  chosen  as  the  model  for  the 
patient.  Figure  4  shows  a  sample  generic  breast  model  before  and  after  adjustment  with 
a  specific  patient  mammogram. 
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(a)  Generic  model  before  using  scaling  (b)  Generic  model  after  comparison  of 

factors  projection  and  target  mammograms 

Figure  4:  Generic  Model  Construction. 


3.2.  Registration 

Image  registration  involves  finding  correspondence  between  coordinates  in  an  im¬ 
age  pair.  If  an  object’s  deformation  can  be  described  by  a  rigid  or  affine  transformation, 
the  correspondence  problem  can  be  readily  solved  using  a  stereo  method  in  an  epipolar 
geometry.  But  the  fact  that  breast  deformation  is  non-rigid  complicates  the  compu¬ 
tation.  For  a  feature  in  one  view,  its  corresponding  epipolar  line  becomes  a  curve 
in  another  view.  Using  a  generic  3D  geometrical  model,  Kita  et  al.  [2]  developed  a 
compression-projection  procedure  to  facilitate  two- view  feature  registration.  We  de¬ 
veloped  a  similar  approach  [27,  28,  29]  with  the  following  improvements:  (1)  A  finite 
element  model  is  superior  to  a  generic  geometrical  model  in  handling  complex  shapes 
and  nonrigid  deformation.  As  a  result,  many  restrictions  associated  with  a  geometrical 
model  can  be  relieved.  For  example,  the  assumption  that  a  breast  only  deforms  within 
the  cross-section  of  CC  view  is  not  needed.  The  assumption  that  a  corresponding  curve 
can  only  deform  uniformly  is  also  waived;  (2)  Compression  parameters  are  corrected 
by  a  global  shape  calibration  to  ensure  the  quality  of  feature  registration. 

In  cases  when  a  lesion  was  visible  in  both  the  CC  and  ML  views,  the  following 
approach  was  taken: 

1 .  The  constructed  natural  3D  breast  model  was  compressed  based  on  the  recorded 
compression  amount  for  one  view  (say  CC  view). 

2.  After  aligning  with  the  view,  based  on  the  direction  recorded  in  the  mammo¬ 
grams,  the  locations  of  all  the  features  (lesions)  identified  in  the  CC  view  were 
back  projected  onto  the  compressed  breast  model  as  straight  lines.  Back  projec¬ 
tion  was  accomplished  through  a  simple  ray  tracing  method.  Figure  5  (c)  and  (d) 
illustrates  this  step. 

3.  The  model  was  adaptively  re-meshed  in  the  regions  adjacent  to  the  straight  lines 
generated  in  step  (2)  and  the  elements  through  which  the  straight  lines  pass  were 
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labeled.  These  features,  which  were  tracked  during  compression,  resulted  in  3D 
curves  (one  for  each  feature)  in  the  uncompressed  breast  volume. 

4.  Steps  (1)  -  (3)  were  repeated  for  each  feature  found  in  the  second  view  (ML). 

5.  A  feature  in  CC  view  was  paired  with  all  features  in  ML  view  and  the  corre¬ 
sponding  distance  between  their  3D  curves  was  measured  (the  distance  between 
two  3D  curves  was  defined  as  the  minimum  distance  between  two  nodes  on  these 
curves). 

6.  All  of  the  feature  pairs  were  ranked  based  on  the  distances  between  their  3D 
curves  in  the  uncompressed  model.  The  pair  with  the  minimum  distance  was 
considered  as  a  match.  In  other  words,  they  are  more  likely  related  to  the  same 
breast  lesion. 

7.  The  average  of  the  two  closest  node  locations  on  these  corresponding  3D  curves 
provided  the  reconstructed  3D  position  of  this  feature  in  the  natural  breast  vol¬ 
ume.  Figure  5  (e)  presents  a  figurative  illustration  of  steps  (5)  -  (7). 

8.  Steps  (5)  -  (7)  were  repeated  for  each  feature  in  CC  view. 


Figure  5:  Correspondence  Recovery  Algorithm. 


Figure  6  shows  an  overview  of  the  system  flow  of  the  proposed  method  including 
all  of  the  pre-processing  and  the  post-processing  procedures. 
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Corresponding  Curves 
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Figure  6:  System  Flow  of  the  Proposed  Approach. 


4.  Experimental  Results 

4.1.  Dataset 

MR  volumes  of  patient  breast  were  used  for  model  construction  and  corresponding 
mammograms  were  selected  for  compression  simulation  experiments.  We  also  used 
phantom  data  for  validation  purposes. 

Patient  Mammograms:  A  dataset  containing  47  mammograms  of  32  patients  (15 
patients  had  2  sets  of  mammograms  scanned  at  different  times)  was  provided  by  the 
Lifetime  Screening  Center,  H.  Lee  Moffitt  Cancer  Center  and  Research  Institute.  Mam¬ 
mograms  were  scanned  at  a  resolution  of  75  micron  and  12  bits  per  pixel,  containing 
mass  or  calcification.  Patient  age  group  ranged  from  30  to  70  years  with  a  median  of 
45  years.  The  images  were  collected  over  a  period  of  time  from  daily  clinical  investi¬ 
gations.  The  center  of  a  lesion  was  determined  as  the  center  of  a  circle  (or  an  ellipse) 
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manually  fitted  by  a  physician  based  on  the  visible  parts  of  the  lesion  in  the  patient 
data.  Table  2  summarizes  the  details  of  the  dataset. 


Patient  Number 

Case  Number 

Lesion  Type 

Age 

Compression  Rate  (mm) 

With  MRI 

1 

1 

Mass 

35-50 

43 

Y 

2 

2 

Mass 

35-50 

46 

N 

3 

3 

Mass 

35-50 

49 

Y 

4 

4 

Mass 

50-70 

52 

N 

5 

5 

Mass 

50-70 

54 

N 

6 

6 

Mass 

35-50 

56 

N 

7 

7 

Mass 

35-50 

44 

N 

8 

8 

Calcification 

50-70 

58 

Y 

9 

9 

Mass 

35-50 

54 

N 

10 

10 

Calcification 

50-70 

63 

Y 

11 

11 

Mass 

35-50 

45 

N 

12 

12 

Calcification 

50-70 

48 

N 

13 

13 

Calcification 

50-70 

60 

Y 

14 

14 

Calcification 

35-50 

52 

Y 

15 

15,16 

Calcification 

50-70 

47,44 

Y 

16 

17 

Calcification 

35-50 

61 

N 

17 

18 

Calcification 

50-70 

47 

N 

18 

19 

Calcification 

35-50 

48 

N 

19 

20,21 

Calcification 

50-70 

43,47 

Y 

20 

22,23 

Calcification 

35-50 

47,46 

Y 

21 

24,25 

Calcification 

50-70 

46,44 

Y 

22 

26,27 

Calcification 

35-50 

45,43 

Y 

23 

28,29 

Calcification 

50-70 

47,48 

Y 

24 

30,31 

Calcification 

35-50 

45,43 

Y 

25 

32,33 

Calcification 

50-70 

52,50 

Y 

26 

34,35 

Mass 

35-50 

44,43 

N 

27 

36,37 

Calcification 

50-70 

49,50 

N 

28 

38,39 

Calcification 

50-70 

45,46 

N 

29 

40,41 

Calcification 

35-50 

43,47 

N 

30 

42,43 

Mass 

50-70 

46,43 

N 

31 

44,45 

Calcification 

35-50 

46,49 

N 

32 

46,47 

Mass 

35-50 

51,50 

N 

Table  2:  Summary  of  the  Dataset 


MR  Volumes  of  Patient  Breast:  The  MR  data  of  patient  breasts  that  was  collected 
for  clinical  investigations  at  the  Lifetime  Screening  Center  was  used  for  model  con¬ 
struction.  The  voxel  size  for  the  MRI  was  1.41  x  1.41  x  2.50  mm.  14  sets  of  patient 
MRIs  were  provided  with  corresponding  mammograms. 

MR  Volumes  and  X-rays  of  Breast  Phantom:  For  phantom  data  experiments, 
we  used  the  Triple  Modality  Biopsy  Phantom  containing  simulated  cystic  masses  and 
dense  masses.  The  phantom  was  scanned  both  with  the  MRI  scanner  and  the  X-ray 
scanner  by  the  Lifetime  Screening  Center  for  algorithm  validation  purpose.  The  Triple 
Modality  Biopsy  Phantom  contained  15  simulated  cystic  masses  and  dense  masses. 
The  voxel  size  for  the  MRI  was  1.41  x  1.41  x  2.50  mm.  Phantom  X-rays  were  scanned 
at  a  resolution  of  75  micron  and  12  bits  per  pixel.  Each  mass  was  mapped  to  a  specific 
element  in  the  finite  element  model  after  model  construction  for  accurate  validation. 
Figure  7  shows  a  sample  phantom  X-ray  image  and  the  corresponding  finite  element 
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model  after  compression  simulation. 


(a)  Phantom  CC  View  X-ray  Image  (b)  Phantom  Model  CC  View  after  Compression 

Simulation 

Figure  7:  Comparison  of  Phantom  X-ray  Image  and  Simulation  Result. 


4.2.  Performance  Evaluation  Measures 

We  designed  several  measures  to  test  the  accuracy  of  the  proposed  method: 

1.  3D  curve  distance:  We  used  the  minimum  distance  between  the  two  3D  curves 
in  the  natural  breast  volume  to  measure  the  registration  error. 

2.  Predicted  3D  position:  After  the  position  of  the  matching  element  was  found 
by  combining  information  from  CC  and  ML  views,  we  calculated  the  Euclidean 
distance  between  the  actual  feature  position  in  the  MR  volume  and  its  prediction 
in  the  model  as  a  measure  of  localization  error. 

3.  Predicted  2D  lesion  position:  We  projected  the  element  set  which  corresponds  to 
the  projection  of  the  feature  point  in  one  view  to  the  other  view,  which  formed 
a  curve.  We  then  calculated  the  minimum  distance  between  the  actual  feature 
point  and  the  projected  curve  locations.  This  measure  of  prediction  error  could 
be  used  for  the  case  when  the  suspicious  object  was  visible  only  in  one  view. 

4.3.  Results  on  Breast  Phantom  Data 

Two  sets  of  phantom  MRIs  were  taken  and  used  for  the  registration  procedure.  One 
was  without  compression  and  the  other  one  with  compression  of  30  mm. 

First,  the  finite  element  model  was  constructed  using  phantom  MRIs  without  com¬ 
pression.  The  Triple  Modality  Biopsy  Phantom  that  we  used  contained  15  simulated 
cystic  masses  and  dense  masses.  After  model  construction,  we  mapped  each  mass  to  an 
element  in  the  finite  element  model.  Then,  we  simulated  the  compression  combining 
the  phantom  material  property,  compression  amount,  and  appropriate  boundary  condi¬ 
tions.  During  the  beginning  of  compression,  each  element  reacted  to  the  displacement 
coming  from  the  neighboring  elements  and  formed  its  own  displacement.  After  the 
compression  simulation  was  fully  completed,  the  model  deformed  and  each  element 
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simulating  the  mass  shifted  to  a  new  position.  We  then  recorded  these  elements’  new 
position  and  compared  them  with  the  actual  position  as  indicated  in  the  MRI  set  with 
compression.  The  experiments  were  repeated  for  all  of  the  15  abnormalities  and  the 
average  error  was  1.6  ±  0.4  mm. 

The  MRI  registration  experiments  above  showed  that  the  finite  element  model 
could  accurately  predict  the  displacement  of  each  inner  element.  Given  this  validation, 
we  then  started  our  experiments  on  finding  correspondence  between  phantom  X-rays. 

To  simplify  the  procedure,  we  used  the  CC  and  the  ML  views  of  the  phantom  as  test 
data.  The  phantom  X-ray  experiment  was  based  on  the  idea  that  feature  points  visible 
in  the  CC  view  and  ML  view  should  be  mapped  to  the  same  point  in  the  uncompressed 
phantom  breast  volume.  We  started  with  the  CC  view.  With  the  mapping  as  shown  in 
Figure  8  (a)  and  (b),  a  feature  point  in  the  CC  view  was  mapped  to  a  set  of  elements. 
The  actual  matching  element  corresponding  to  the  projected  simulated  mass  in  the  CC 
view  was  among  these  elements.  Further  information  was  gathered  from  the  ML  view 
which  was  used  to  complete  the  matching  in  the  following  manner.  The  same  feature 
point  (feature  matching  was  done  based  on  the  distances  between  their  corresponding 
3D  curves)  in  the  other  view  was  identified  and  another  set  of  elements  which  corre¬ 
sponded  to  the  projection  in  the  ML  view  was  chosen.  Now  we  had  two  compressed 
models  corresponding  to  the  CC  and  ML  view  projections  and  two  sets  of  elements 
corresponding  to  the  same  feature  point  in  the  projections.  These  features,  which  were 
tracked  during  compression,  resulted  in  3D  curves  in  the  uncompressed  breast  volume. 
The  two  sets  of  elements  were  then  put  together  in  the  same  space.  The  mean  location 
of  the  two  closest  nodes  on  these  curves  gave  us  the  element  that  corresponded  to  the 
actual  feature  point. 


(a)  Phantom  MRI  (b)  FE  Model  projection  (c)  Phantom  MRI  after 

before  compression  compression 


Figure  8:  Feature  point  mapping  in  phantom  data. 

For  validation  purposes,  we  selected  a  set  of  feature  points  visible  in  both  views 
in  the  phantom  data.  First,  the  predicted  position  is  computed  (the  deformed  projected 
curve),  then  the  minimum  Euclidean  distance  between  the  real  feature  position  and  its 
prediction  is  calculated  as  an  indicator  for  accuracy.  The  average  distance  between  the 
3D  curves  of  the  corresponding  features  was  1.6  mm.  The  average  error  in  localization 
was  2.1  mm.  The  average  error  in  the  natural  breast  volume  with  simulated  compres¬ 
sion  was  within  2.6  mm.  Table  3  summarizes  the  results  of  the  proposed  method  on 
the  phantom  data.  The  accuracy  of  the  finite  element  model  we  constructed  is  clearly 
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evidenced  in  this  experiment. 


Average  Error 

(mm) 

Average  Feature  Di¬ 
ameter  (mm) 

Average  Feature  Dis¬ 
tance  (mm) 

3D  curve  distance 

1.6  ±0.4 

10-20 

25.9  -  80.0 

Predicted  3D  lesion  position 

2.6  ±  0.8 

10-20 

25.9  -  80.0 

Predicted  2D  lesion  position 

2.1  ±0.4 

10-20 

25.9  -  80.0 

Table  3:  Results  on  Phantom  Data  (Simulated  Mass  and  Calcification) 


4.4.  Results  on  Patient  Data  using  Patient  Specific  Models 

The  localization  accuracy  of  the  registration  procedure  was  evaluated  using  22  clin¬ 
ical  patient  cases  (14  patients).  Breast  model  was  constructed  from  each  patient’s  MR 
images.  For  patients  who  had  mammograms  collected  at  two  different  times,  for  the 
second  time,  the  model  was  picked  from  the  first  session  but  adjusted  for  physiolog¬ 
ical  changes  using  Equations  2,  3,  and  4.  The  minimum  3D  distances  for  projected 
curves  were  computed  to  validate  the  model  and  the  algorithm.  The  average  minimum 
distance  was  2.22  mm  and  the  average  error  of  predicted  2D  lesion  position  was  3.03 
mm.  Table  4  summarizes  the  results  of  the  proposed  method  on  real  patient  data  using 
patient  specific  models.  These  numbers  are  well  within  the  required  accuracy  limits  for 
reliable  clinical  diagnosis. 


3D  curve  distance 
(mm) 

Predicted  2D  lesion  po- 
sition(mm) 

Feature  Diameter 
(mm) 

Case  1 

1.6 

3.1 

7.2 

Case  3 

1.2 

2.1 

8.1 

Case  8 

1.5 

2.6 

3.5 

Case  10 

0.4 

1.6 

5.4 

Case  13 

1.5 

2.4 

3.8 

Case  14 

1.7 

2 

5.2 

Case  15 

1.3 

2.5 

7.3 

Case  16 

3 

3.6 

5.4 

Case  20 

2.1 

2.7 

7.1 

Case  21 

3.1 

3.8 

6.9 

Case  22 

2.3 

3 

5.3 

Case  23 

3.1 

3.7 

5.6 

Case  24 

1.9 

2.9 

5.6 

Case  25 

2.2 

3.4 

5.9 

Case  26 

2.2 

3.1 

4.8 

Case  27 

3 

3.7 

4.7 

Case  28 

2.3 

2.9 

7.6 

Case  29 

3.1 

3.7 

7.9 

Case  30 

1.9 

3 

8.8 

Case  31 

3.8 

3.5 

8.5 

Case  32 

2.3 

3.2 

7.5 

Case  33 

3.3 

4.1 

7.8 

Average  (Std.  Deviation) 

2.22  (±  0.63) 

3.03  (±  0.48) 

Table  4:  Results  on  Patient  Data  using  Patient  Specific  Models 


The  groundtruth  was  generated  by  a  radiologist  who  identified  the  lesion  location  in 
both  the  CC  and  the  ML  views.  We  performed  feature  registration  using  the  proposed 
algorithm.  Figure  9  shows  this  step  on  a  sample  real  data.  The  feature  identified  in  CC 
view  is  shown  as  a  black  cross  in  Figure  9  (a).  The  feature  was  then  back-projected 
onto  the  compressed  3D  model  and  its  corresponding  straight  line  (blue)  is  shown  in 
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Figure  9  (b).  After  model  restoration  to  original  uncompressed  state,  the  straight  line 
(blue)  deformed  into  a  curve  (red).  Note  that  the  blue  line  and  the  red  curve  are  inside 
the  3D  model.  The  3D  model  was  compressed  again  in  the  ML  direction.  Finally,  the 
curve  was  projected  onto  the  ML  imaging  plane  as  shown  in  Figure  9  (c).  In  Figure  9 
(d),  the  projected  curve  was  overlaid  with  the  ML  view  mammogram.  It  can  be  seen 
that  the  feature  identified  in  the  ML  view  (black  cross)  is  located  on  the  projected  curve, 
indicating  that  the  two  features  identified  in  CC  and  ML  views  are  highly  related.  It 
should  be  pointed  out  that  even  with  the  adaptive  meshing,  the  element  size  is  still 
larger  than  the  feature  size  (less  than  5  mm  in  diameter),  which  explains  the  width  of 
the  projected  curve  in  Figure  9  (c)  and  (d).  A  finer  mesh  is  probably  needed  in  a  biopsy 
design,  at  the  cost  of  a  much  larger  number  of  elements  and  subsequent  computational 
complexity.  Generally,  an  exact  intersection  of  the  approximated  curves  is  not  expected 
because  in  any  registration  procedure  errors  are  bound  to  be  present.  Therefore,  the 
lesion  position  is  approximated  to  lie  at  the  center  of  the  line  connecting  two  points  of 
the  curves  with  the  smallest  distance. 


(c)  (d) 

Figure  9:  Illustration  of  the  feature  prediction  procedure  on  a  sample  patient  data. 

4.5.  Results  on  Patient  Data  using  Generic  Model 

We  tested  on  47  patient  mammograms  that  had  no  associated  MR  data  to  show  the 
accuracy  of  the  correspondence  recovery  with  a  generic  model.  After  the  3D  curve 
calculation,  the  position  where  the  two  3D  curves  intersect  each  other  served  as  the 
prediction  of  the  lesion  when  the  model  is  restored  to  its  original  shape.  We  could  then 
use  the  element  at  that  position  in  the  3D  volume  to  track  the  lesion  position  during 
the  compression.  Table  5  summarizes  the  results  of  the  proposed  method  using  generic 
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models. 


Case 

3D  curve  dis- 

tance(mm) 

Predicted  2D  lesion  po- 
sition(mm) 

Feature  Diameter 
(mm) 

Case  1 

3.0 

4.7 

7.2 

Case  2 

5.3 

3.0 

10.1 

Case  3 

4.1 

4.3 

8.1 

Case  4 

3.9 

4.9 

7.2 

Case  5 

3.5 

6.2 

5.2 

Case  6 

4.1 

4.7 

2.6 

Case  7 

4.3 

3.6 

5.1 

Case  8 

4.6 

5.9 

3.5 

Case  9 

4.0 

3.2 

2.9 

Case  10 

3.8 

4.2 

5.4 

Case  1 1 

3.8 

4.6 

3.9 

Case  12 

4.7 

5.3 

4.2 

Case  13 

3.7 

4.6 

3.8 

Case  14 

4.2 

4.8 

5.2 

Case  15 

4.5 

4.9 

7.3 

Case  16 

4.6 

4.7 

5.4 

Case  17 

4.5 

5.3 

5.3 

Case  18 

3.8 

2.4 

7.2 

Case  19 

3.6 

2.8 

3.9 

Case  20 

3.2 

4.2 

7.1 

Case  21 

3.9 

4.4 

6.9 

Case  22 

4.0 

5.3 

5.3 

Case  23 

3.6 

4.6 

5.6 

Case  24 

3.4 

4.3 

5.6 

Case  25 

4.1 

4.5 

5.9 

Case  26 

4.2 

5.2 

4.8 

Case  27 

3.5 

4.7 

4.7 

Case  28 

4.7 

3.8 

7.6 

Case  29 

5.1 

6.2 

7.9 

Case  30 

4.4 

3.4 

8.8 

Case  31 

4.2 

4.4 

8.5 

Case  32 

4.2 

4.8 

7.5 

Case  33 

5.2 

5.6 

7.8 

Case  34 

4.1 

4.8 

6.7 

Case  35 

4.6 

5.0 

6.2 

Case  36 

5.0 

5.1 

8.2 

Case  37 

5.1 

4.9 

8.1 

Case  38 

5.0 

5.6 

7.6 

Case  39 

4.6 

3.9 

7.4 

Case  40 

4.9 

6.4 

8.9 

Case  41 

4.3 

3.5 

8.5 

Case  42 

4.1 

4.6 

7.9 

Case  43 

4.1 

5.0 

8.2 

Case  44 

5.0 

5.8 

8.1 

Case  45 

4.0 

5.0 

8.3 

Case  46 

4.5 

5.2 

7.1 

Case  47 

4.7 

5.3 

6.9 

Average  (Std.  Deviation) 

4.2  (±  0.44) 

4.7  (±  0.65) 

Table  5:  Results  on  Patient  Data  using  Generic  Model 


Table  6  compares  the  accuracy  results  of  our  method  with  some  of  the  existing 
registration  methods  in  literature. 
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Method 

Patient  dataset  size 

3D  curve  distance 

Predicted  2D  le¬ 
sion  position 

Ruiter  et  al.  (2006)  [30] 

6 

3.9  ±1.7  mm 

4.3  ±1.0  mm 

Our  method  (Patient 
Specific  Model) 

22 

2.22  ±  0.63 

3.03  ±  0.48 

Our  method  (Generic 
Model) 

47 

4.2  ±  0.44 

4.7  ±  0.65 

Table  6:  Comparison  with  Exisiting  Registration  Methods 


4.6.  Analysis 

There  are  many  studies  in  the  literature  for  measuring  the  elasticity  parameters  of 
soft  tissues  [41,  12,  20].  According  to  these  studies  the  average  Young’s  modulus 
value  for  fatty  tissue  is  lKPa,  for  skin  is  88KPa,  and  for  glandular  tissue  is  lOKPa. 
The  overall  Young’s  modulus  is  considered  to  lie  in  the  range  5KPa  -  15KPa  for  the 
entire  breast  modeled  as  a  linear,  continuous,  incompressible,  isotropic,  and  homoge¬ 
nous  tissue.  Since  we  are  interested  in  constructing  a  generic  model  we  chose  an  initial 
Young’s  modulus  value  of  lOKPa  [39].  Since  the  breast  is  considered  to  be  an  incom¬ 
pressible  tissue,  theoretically  volume  is  preserved  for  a  Poisson’s  ratio  value  of  0.5. 
However,  high  Poisson’s  ratio  can  lead  to  instabilities  in  the  FE  model.  A  value  be¬ 
tween  0.490  -  0.495  is  generally  accepted  as  computational  stable  yielding  minimum 
displacement  error.  After  model  calibration,  we  used  a  Poisson’s  ratio  value  of  0.490. 

Further,  for  the  same  settings  for  plate  compression,  Young’s  Modulus,  and  Poisson 
ratio,  we  achieved  similar  accuracy  for  two  different  patient  age  groups  (Table  7).  This 
illustrates  that  the  assumed  Young’s  Modulus  and  Poisson  ratio  were  suitable  for  cases 
across  different  age  groups. 


Age  (years) 

Average  3D  curve 
distance  (mm) 

Average  Predicted  2D 
lesion  position  (mm) 

[35,50) 

[  50,  70 ) 

4.2  ±  0.38 

4.3  ±  0.50 

4.6  ±  0.70 

4.8  ±0.61 

Table  7:  Generic  modeling  results  on  two  different  patient  age  groups  (Young’s  Modulus  =  lOKPa,  and 
Poisson  ratio  =0.495). 

With  larger  compression  rates,  the  error  rate  for  the  calcification  cases  increased 
but  not  significantly.  This  is  due  to  the  fact  that  when  plate  compression  increased,  the 
area  of  the  resulting  patient  mammogram  also  increased,  thus  leading  to  a  larger  scaling 
factor  and  changes  to  the  shape  of  the  generic  model.  Table  8  presents  the  localization 
error  of  the  method  for  large  compression  rates. 


Plate  Compression  | 

[  43,  47  ]  mm 

[  48,  54  ]  mm 

[  55,  63  ]  mm 

Average  3D  curve  Distance  (mm) 
Average  2D  Localization  (mm) 

3.9  ±  0.45 

4.3  ±  0.56 

4.0  ±  0.29 

4.5  ±  0.91 

4.1  ±0.33 

4.9  ±  0.53 

Table  8:  Performance  on  calcification  cases  using  generic  mode  with  different  plate  compression. 


5.  Conclusions 

Detecting  micro-calcifications  and  masses  in  mammograms  taken  from  two  differ¬ 
ent  views  is  critical  for  early  diagnosis  of  breast  cancer  and  successful  treatment.  We 
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presented  a  deformable  model  based  method  to  improve  feature  registration  between 
two  mammographic  views.  We  built  a  finite  element  model  of  the  breast  based  on 
real  patient  MR  data.  The  advantage  of  using  a  3D  finite  element  model  is  that  non- 
rigid  breast  deformation  can  be  computed  accurately,  which  is  lacking  in  2D  registra¬ 
tion  methods.  We  devised  a  stepwise  incremental  approach  to  simulate  plates  motion, 
which  enables  us  to  model  large  breast  deformation  through  a  series  of  static  equi¬ 
librium  calculation.  We  also  employed  an  adaptive  meshing  technique  to  reduce  the 
computational  cost.  The  use  of  MRIs  of  the  same  patient  to  build  a  finite  element 
model  further  ensures  the  registration  quality.  The  following  steps  summarizes  the 
steps  involved: 

1.  3D  Finite  element  model  of  the  breast  was  constructed  using  breast  MR  images. 

2.  Breast  model  was  compressed  using  recorded  compression  and  direction  data  in 
the  mammograms. 

3.  Features  (calcification  and  mass)  were  manually  identified  by  radiologists  in  two 
mammographic  views. 

4.  Identified  mammographic  features  were  back-projected  onto  the  model  to  gen¬ 
erate  their  3D  positions  in  the  natural  breast  volume. 

5.  Correlation  of  mammographic  features  from  two  views  was  determined  based 
on  their  3D  positions. 

Results  presented  in  this  paper  showed  that  our  approach  could  predict  lesion  cor¬ 
respondence  with  reasonable  accuracy.  A  dataset  containing  22  patient  mammograms 
along  with  associated  MR  data  was  built  and  tested  with  patient  specific  models. 

In  the  absence  of  MR  data,  a  generic  modeling  approach  allowed  simulation  of  bio¬ 
logical  soft  tissue  deformation  by  choosing  and  adjusting  an  alternative  model  that  best 
corresponds  to  a  patient’s  mammogram.  Experimental  results  on  a  dataset  consisting 
of  47  mammograms  allowed  us  to  demonstrate  that  this  model  succeeds  in  reproducing 
the  mechanical  behavior  of  breast  tissue  in  a  compression  experiment  with  reasonable 
accuracy. 

As  future  work,  we  plan  to  extend  this  method  to  cases  when  the  lesion  is  visible 
in  only  one  view.  We  propose  to  adopt  the  following  approach  to  find  the  predicted 
position  for  the  same  lesion  in  the  other  view: 

1.  The  view  where  a  suspicious  feature  was  found  (say  CC  view)  is  chosen  as  the 
starting  point.  The  3D  finite  element  model  is  compressed  with  the  recorded 
compression  amount  and  direction  in  the  CC  view. 

2.  The  feature  is  back-projected  onto  the  compressed  model  to  generate  a  straight 
line. 

3.  The  model  is  adaptively  re-meshed  in  the  regions  adjacent  to  the  straight  line 
generated  in  step  (2)  and  the  elements  through  which  the  straight  line  passes  is 
labeled.  The  detected  feature,  which  is  tracked  during  compression,  results  in  a 
3D  curve  in  the  uncompressed  breast  volume. 

4.  The  model  is  then  compressed  with  the  recorded  ML  data  and  the  curve  found  in 
the  original  model  in  step  (3)  deforms. 
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5.  After  complete  compression  simulation,  the  deformed  curve  is  projected  onto 
the  view  where  no  lesion  was  found  in  the  beginning.  The  region  along  this 
curve  indicats  the  most  suspicious  area  in  which  a  second  reading  is  strongly 
recommended  to  see  whether  a  featured  was  overlooked.  The  thickness  of  this 
2D  curve  is  determined  by  the  resolution  of  the  3D  model  and  the  size  of  the 
feature  identified  in  the  first  view  (CC). 

A  subsequent  direction  we  have  attempted  to  explore  is  in  correspondence  identifi¬ 
cation  between  image  features  identified  in  two-view  mammography  taken  at  different 
times  [42,  43].  We  use  the  3D  finite  element  model  as  a  tool  for  simulating  and  ana¬ 
lyzing  breast  deformation  to  improve  the  accuracy  of  matching  in  temporal  mammo¬ 
grams  [44]. 

In  conclusion,  we  believe  that  the  methods  presented  in  this  paper  offer  useful  tools 
for  lesion  position  prediction  and  lesion  correspondence  construction.  They  hold  great 
promise  in  both  early  breast  cancer  diagnosis  and  the  subsequent  surgery  planning. 
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ABSTRACT 

Accurate  matching  and  measurement  of  breast  lesions  identified  on  multiple  temporal  mammographic 
views  of  breast  are  vital  in  detecting  and  treating  breast  cancer.  However,  lack  of  3D  structural 
knowledge  and  large  compression  of  breast  during  x-ray  imaging  often  cause  mismatch  among 
temporal  mammograms,  resulting  in  incorrect  diagnosis  or  localization.  A  3D  model  is  strongly 
desired  to  provide  accurate  information  about  a  breast's  3D  geometry  as  well  as  its  deformation.  We 
have  developed  a  3D  biomechanical  model  to  simulate  and  analyze  breast  deformation  and  to  register 
breast  lesions  on  multiple  views  of  mammograms.  This  study  is  expected  to  significantly  improve  the 
accuracy  of  matching  in  temporal  mammograms  and  thus  the  performance  of  diagnosis  and 
treatment. 

Two  types  of  3D  biomechanical  models  of  breast  deformation  have  been  developed  and  numerically 
simulated  by  using  rigorous  finite  element  method  (FEM).  One  model  is  developed  upon  magnetic 
resonance  (MR)  images  of  the  breast,  and  the  other  one,  named  generic  model,  is  based  solely  on 
mammograms.  When  available,  MR  images  of  breast  are  used  to  construct  natural  breast  shape. 
When  MR  images  of  breast  are  not  available,  we  utilize  generic  model  of  breast  deformation  based  on 
breast  volume  estimate.  Through  numerical  simulations  of  breast  deformation,  it  is  observed  that  a 
spot  on  mammogram  corresponds  to  a  curve  in  the  natural  breast  volume.  Based  on  this,  we  have 
developed  matching  methods  to  locate  the  same  breast  lesions  in  multiple  views  of  mammograms  by 
minimizing  distances  between  recovered  curves  from  the  different  views  of  mammograms. 

We  have  experimented  with  the  proposed  methods  on  breast  phantom  images  and  clinical 
mammograms;  patient  images  contain  both  cases  with  or  without  simultaneous  MR  images.  The 
biomechanical  model  of  breast  deformation  and  registration  of  breast  lesions  on  one  time 
mammograms  have  been  well  validated.  Currently,  we  are  working  on  registration  among  multiple 
temporal  mammograms. 

This  study  proposes  to  use  a  3D  finite  element  model  with  routine  clinical  data  to  assist  detection  and 
treatment  of  breast  cancer.  We  have  been  incorporating  principles  of  radiology,  breast  imaging,  and 
computer  vision  to  the  research  for  accurate  detection,  location,  and  corresponding  treatment  of 


breast  cancer.  Moreover,  the  proposed  modeling  technique  can  be  used  in  routine  screening, 
diagnosis,  and  surgical  planning. 
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